Definition
Some aspects and outcomes of choice cannot be tested, leading to unrealistic confidence in judgement.
Classification reasoning
There are plenty of testing frameworks and the concept of test driven development is followed by large number of teams. Establishing a sound test strategy ensures a stable code base at all times and reduces failures.
Example
Automated frontend testing: Achieving hundred percent code coverage is generally tricky. It is hard to test all aspect of frontend code using automated test frameworks. Realistic expectations have to be set, and it is hard to measure how much testing is realistic. Testing complex corner cases are sometimes avoided if the user interface looks right and works as expected in most cases.
Impact
As the user interface would work as expected in most cases, it leads to an unrealistic confidence on the test code. It would be harder to detect failures during future deployments as the tests would pass even when there could be a potential error.
Debiasing techniques
Identify a testing strategy and establish a testing philosophy. Use test driven development where possible. Always try to keep the tests up to date. Document cases which are hard to test as these could indicate areas of failure in the future.