[Bug] Tests wait on a fixed delay instead of the observable outcome #513

New issue

Closed

opened 2026-07-22 09:28:23 +00:00 by brenno · 0 comments

brenno commented

2026-07-22 09:28:23 +00:00

Owner

Four tests were repaired in a single day with the same defect: they wait on real disk or isolate work using a fixed Future.delayed inside runAsync, instead of waiting for the observable outcome.

test/shell_export_actions_test.dart — twice. The wait ended when the file appeared, but the assertion was on the confirmation message, which is only built once the rest of the future completes.
test/signature_draw_test.dart
test/duplicate_cleanup_dialog_test.dart — fixed 50 ms and 100 ms sleeps

They all pass in isolation and fail under a loaded make check. That is the worst failure mode: the test is re-run, passes, and everyone concludes it was a fluke. It costs an hour of looking for a bug that is not there, every busy day.

Proposed guard. This pattern is greppable: a Future.delayed with a constant duration inside a runAsync block in test/. A check in tool/ that fails on it, with a baseline that can only shrink, would stop the habit spreading. The replacement is a bounded pump-until-condition helper with a time limit, so genuinely stuck I/O still fails the test instead of hanging the suite — settleAsync in shell_export_actions_test.dart and _wachtTot in duplicate_cleanup_dialog_test.dart are two worked examples.

Worth checking whether more instances exist before building the guard: the four above were found incidentally, not by searching.

Four tests were repaired in a single day with the same defect: they wait on real disk or isolate work using a **fixed `Future.delayed`** inside `runAsync`, instead of waiting for the observable outcome. - `test/shell_export_actions_test.dart` — twice. The wait ended when the file appeared, but the assertion was on the confirmation message, which is only built once the rest of the future completes. - `test/signature_draw_test.dart` - `test/duplicate_cleanup_dialog_test.dart` — fixed 50 ms and 100 ms sleeps They all pass in isolation and fail under a loaded `make check`. That is the worst failure mode: the test is re-run, passes, and everyone concludes it was a fluke. It costs an hour of looking for a bug that is not there, every busy day. **Proposed guard.** This pattern is greppable: a `Future.delayed` with a constant duration inside a `runAsync` block in `test/`. A check in `tool/` that fails on it, with a baseline that can only shrink, would stop the habit spreading. The replacement is a bounded pump-until-condition helper with a time limit, so genuinely stuck I/O still fails the test instead of hanging the suite — `settleAsync` in `shell_export_actions_test.dart` and `_wachtTot` in `duplicate_cleanup_dialog_test.dart` are two worked examples. Worth checking whether more instances exist before building the guard: the four above were found incidentally, not by searching.

brenno added the

accepted

bug

labels

2026-07-22 09:28:23 +00:00

brenno referenced this issue from a pull request that will close it,

2026-07-22 10:44:02 +00:00

feat: bewaak dat een test op het resultaat wacht en niet op de klok #530

brenno referenced this issue from a commit

2026-07-22 10:44:02 +00:00

feat(poort): geen vaste vertraging binnen runAsync in tests

brenno closed this issue