Unit Tests Don't Find Bugs: the Death of QA

I blasted my cache and cookies away...all is well.  Strange stuff...I wasn't having problems with any other sites.

There are more benefits to TDD besides regression testing.

Practicing TDD encourages separation of concerns, I could go on.

I consider the biggest benefit of TDD to be that the tests are "first consumer" of the code.

This is not a flame, just a reminder that TDD is not strictly about preventing regressions.  In my opinion the regression testing benefit is secondary.

you're absolutely right about TDD not helping you find all the bugs in your software, but i'm flogging a dead TDD horse when i say that TDD is primarily meant to improve the design of your software, not provide test coverage (a useful by-product), so when (not if) you find those bugs in your code, you will hopefully be able to isolate it to the 5-line method in a small, well-designed single responsibility (more or less) class, which caused it, instead of a 500 line method in a 2000 line class doing everything.

If the only way you can keep from writing 500 line methods and 2000+ line class files is via TDD, you have much bigger problems on your hands.  Ironically, the people who are really inclined to do TDD are the ones that need it the least.

Agreed - unit tests and testers find different sorts of problem. We write a lot of unit tests, but we also have a lot of testers (about 1 tester for every 1 – 2 developers) who are essential for finding ui glitches, workflow problems, etc. We separately use an automated error reporting system to let users send in details if an application crashes, which helps us find the bugs which unit testing is unlikely to catch and which get missed by QA. It means we usually find out about any nasty bugs within a few days of shipping and can fix it a lot faster than waiting for customers to complain. A tester I work with wrote about it at http://www.simple-talk.com/blogs/2011/09/08/smartassembly-error-reporting-with-the-bug-finding-power-of-more-than-20-testers/ 

If you have a problem with failing unit tests then you need unit tests.  If QA testers are finding real bugs or customers are reporting bugs then you need QA testers.  If the quality of the design/code is a problem, then deal with it at that level.  Do what works in your environment to catch bugs early.

TDD is not the absolute solution to detect bugs. The best solution so far is to integrate QA into the developer team. QA can then do the exploratory testing, and the developer can assist them really early in the development cycle. And when the reproducible datasets are already there, it's a breeze to automate the tests scenarios. Of course you cannot detect every possible issue, but you can try to detect most.

A Scrum teacher told people in my company that the developers are the most qualified persons to test their own code. I say bullshit. A pressured developer is focused on achieving the main, written features. Most developers I've been working with are not focused on details and we cannot always foresee what the end users, or the external systems will do in real life. We imagine secondary scenarios, but these are not often the right ones.Any external, non-developer early feedback and ideas are highly welcomed and valuable.

I don't mean to whinge, but how the heck are you meant to unsubscribe from a thread?  When I click on the link it says "Access Denied", and I've had no reply from the DZone email address :(

OMG, NO! Manual QA is a monstrosity that systematically cripples your ability to create a high frequency feedback loop with customers. Aggressive continuous deployment pipelines will destroy the manual testing model in a side-by-side competition every time. When you start piling up manual tests, you'll be forced you into longer and longer release cycles. If you aren't capable of releasing multiple times a day, you are inevitably heading down a road to producing crappy, expensive software that would wilt under real competition. 

It sounds like your problem is that you aren't doing TDD well. Unit tests are once facet of TDD, but if that's all you are relying on, you're crippling your efforts. The TDD strategy that high velocity continuous delivery shops use is a mixed BDD and unit testing strategy. BDD tests the user stories as stated in the voice of the customer. Unit tests are a secondary form of testing which have an internal developer focus: they assure APIs are clean, they prevent regressions, and they enable refactoring. Fundamentally, though, it's the BDD tests that achieve quality. 

Most "bugs" actually don't violate any perviously stated acceptance criteria, and so we should view fixing them as a prioritization decision against competing features. The actual users of the software will make better decisions than QA people who live in a bubble every time on this. There should only be one voice of the customer giving input on prioritization and if that isn't literally the customer, you've created a strategic deficiency in your software development process. 

If you ship lots of tiny releases and have good feedback mechanisms from users, and use feature flags to confine exposure of new functionality, you'll have an experience sooner or later where you ship something that breaks unstated needs, and you add the acceptance criteria or story and it gets prioritized to "right now" and you fix it same day while the feature flag is still set to the beta users group. After this happens, you'll look on big QA as the wasteful monstrosity that it is. Nobody that I know that has tried this model ever goes back.