In a recent discussion about whether Agile is dead or not, I read an interesting comment:

I've seen this cycle a few times. Let me handwave how it goes:
  • Agile Critisism: The snake oil is all over and getting worse!
  • Agilista: It's not done properly!
  • Agile Criticism: No True Scotsman!
pnathan on Hacker News

Indeed, I too have seen this discussion happening. So, are we really using the “No True Scotsman” fallacy when we say that failed agile projects were not done properly?

We commit the “No True Scotsman” fallacy when we exclude counter examples from our theory (or statement) by modifying the theory (statement):

Person A: "No Scotsman puts sugar on his porridge." Person B: "I am Scottish, and I put sugar on my porridge." Person A: "Well, no true Scotsman puts sugar on his porridge." No True Scotsman on Wikipedia</blockquote> So, if I say "Any agile project that failed was not done properly", it would be true: I would be committing this fallacy. But I don't think this is the case in many of those discussions. Many of the discussions/blog posts start like this:
  • The snake oil is all over and getting worse! Look at project X. They used methodology Y and tools A, B and C, and they failed horribly!
  • Every methodology Y project I did was the horror. methodology Y is just an excuse for managers to micro-manage.
  • ...
In that case, if I can find specific reasons why those projects do not fit the definition of agile software development, I do not commit the "No True Scotsman" fallacy. I merely reject your example, I do not modify my statement about agile or theory of agile. Now, I know that there is no definition of agile. But there are some things that come close to a definition. Just assess the project under discussion (or your current project) against one of them. For example, a very simple and powerful definition is "agile as a doctrin": Reduce the distance between problems and problem-solvers: How far is your development team separated from actual users? Can they talk to them? How far are your team and the actual users separated from budget decisions? Product initiation Decisions? Have you really reduced the distance between problems and problem solvers? Validate every step: Do all your user stories really reflect customer needs? Does the business reason in your stories really reflect customer goals? Does every sentence on the story card add value? Does no sentence on the story card constrain the technical solution? Does every part of your solution address the needs / move you toward the goals? Are your tests adding value? Are all your deployments correct? Are you really validating every step? Take smaller steps: How many user stories are in your backlog? Could you get away with less? How often do you meet with real users? Can you do it more often? How large are your user stories? Can you make them smaller? How often do you deploy to test and live environments? Can you do it more often? How long are your budgeting / planning cycles? Can you make them shorter? Are you really taking smaller steps? Clean up as you go: Do you always refactor you code when all the tests are running? Do you always refactor your tests? Do you delete redundant / outdated tests? Do you regroup acceptance tests by feature set after stories are done? Do you keep your backlog short and sorted? Do you re-write/improve user stories? Do you remove meetings/artifacts/procedures from your process when they do not add value? When was the last time you edited your definition of done? Are you really cleaning up as you go? So, if you are not really reducing the distance between problems and problem solvers, if you are not really validating every step, if you are not really taking smaller steps or if you are not really cleaning up as you go, then you are not doing Agile as good as you can (according to this definition). And if your project fails because of this [1], I will point it out. (And the questions show you where you could try to improve.) The Agile Manifesto is another definition you could assess your project against. It only contains four values and twelve principles. Just ask yourself: Are we really valuing those four things? Do we really follow those twelve principles? If you assess projects like this, you'll probably realize that many projects that claim to be agile have a lot of room for improvement. And some of these projects don't really try to improve. So, are they really agile? Many of the "agile is snake oil" stories are in this category. On the other hand, you might realize that some projects who don't claim to be agile are in fact doing pretty well when assessed against those definitions. Should we really count them as "not agile" in the discussions? Maybe we really sometimes hear "No True Scotsman" arguments in the ongoing discussions about agile software development and the death of "Agile". But often, the kinds of projects being discussed arguably were not doing agile properly. When we call that out, we are not committing a "No True Scotsman" fallacy. We are just rejecting an incorrect example. [1] Ok, I know it's hard (or even impossible) to know exactly why a project has failed. Anyway. I have written about "doing agile properly" in the past. You might be interested in my articles Scrum... But? and ScrumBut... and the long run. You might be also interested in: Learn more about how I can help you save money and earn money by improving your agile software development process.