Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trying to understand the output from the failing Nash equilibrium tests recently just about did my head in, so I decided it was high time that we really need to look at our tests. So I pulled together a bunch of notes/thoughts I had been having (alongside reading up on some
pytestfeatures)..This draft pull request is a proposal for discussion.
In short - I have re-written what used to be the test for
enummixed_solvewith rational probabilities to be a generic Nash equilibrium solver tester (!!!)Highlights:
dataclasstest1,test2&c right now, they will need to be better in production of course.pytest-subtests(which has been added as a dependency in the testsrequirements.txt). This allows us to isolate the individual checks within the test so even if one fails the others can still run - e.g. if somehowmax_regretis correct but the probabilities change for one profile, it still will check them all.Q()as a shorthand for rationals just to make our intent precise, and a helperd()which visually represents a probability distribution - solely for visual layout so we are not lost in a maze of square brackets, all alike.This is an all hands request for everyone to have a look. Not least because in the new year I will be asking everyone to kick in for rationalising our test suite using either this technique or some other one - it is clear our test suite's reaching the edge of maintainability without making some investments like the ones suggested here!