README.md (6568B)
1 # Ratchets 2 3 Tests that lazily enforce a requirement across the entire repository. 4 5 # What is it? 6 7 Ratchets is a lazy way to enforce code compliance on an ongoing basis. This is done by defining regular expressions and shell commands to run against all non-excluded python files in a given repository. Tests only fail when the number of non-compliant lines of code increases. This ensures future code does not have bad patterns, while allowing old code to coexist until it is phased out. 8 9 # Installation 10 11 ## Required 12 13 ```bash 14 pip install ratchets 15 ``` 16 17 ## Optional 18 19 **Note:** This is only required if you plan to use Ratchets with PyTest. 20 21 ```bash 22 pip install pytest 23 ``` 24 25 # Usage 26 27 First, create a tests.toml file at the root of your repository. See [tests.toml](https://github.com/andrewlaack/ratchets/blob/main/tests.toml) for an example of how this should look. There are two primary rule types that can be defined in the tests.toml file. 28 29 ## ratchet.regex 30 31 These are tests that check regular expressions for each line of code in each file being examined. 32 33 **Example:** 34 35 ```toml 36 37 [ratchet.regex.exceptions] 38 regex = "except:" 39 valid = [ 40 """try: 41 x = 1 42 except ValueError: 43 pass""", 44 """try: 45 do_something() 46 except (IOError, ValueError): 47 handle()""" 48 ] 49 invalid = [ 50 """ 51 try: 52 pass 53 except: 54 pass""", 55 """try: 56 dangerous() 57 except: 58 recover()""" 59 ] 60 description = "Bare except clauses catch all exceptions indiscriminately. This can hide bugs and important exceptions. To mitigate this, explicitly state the exception types that will be handled in the except clause." 61 62 ``` 63 64 The valid and invalid entries are not necessary, but we provide a CLI utility to verify the regular expressions don't exist in the valid strings and do exist in the invalid strings. This can be ran with: 65 66 ```bash 67 68 python3 -m ratchets.validate 69 70 ``` 71 72 If you are testing a .toml file that is not the repository default, it can be specified with: 73 74 ```bash 75 76 python3 -m ratchets.validate -t FILENAME 77 78 ``` 79 80 The description entry is also optional, but if provided, it will be included in the output of failing PyTest tests. 81 82 ## ratchet.shell 83 84 These are tests that run against each file where each evaluation is of the form: 85 86 ```bash 87 FILEPATH | SHELL_COMMAND 88 89 ``` 90 91 The standard output of the command is assumed to describe infractions, and the number of lines dictates the total number of infractions. It should also be noted that internally we perform a lookup for the line number based on the standard output. As such, ensure the standard output is the **exact** same text from the line that contains infractions. 92 93 **Example:** 94 95 ```toml 96 97 [ratchet.shell.line_too_long] 98 command = "xargs -n1 awk 'length($0) > 88'" 99 description = "Black sets the max line-width to 88 to help with the readability of code. Ensure all lines have <89 characters. You can run 'black FILENAME' to fix this issue." 100 101 ``` 102 103 This is an example of an `awk` command being used to print each line that has more than 88 characters (this is the default line-length for [black](https://github.com/psf/black)). As these are printed, they are counted as infractions. 104 105 ## Updating Ratchets 106 107 Once your rules are defined, you need to count the infractions. This is done by running: 108 109 ```bash 110 python3 -m ratchets -u 111 ``` 112 113 This creates a ratchet_values.json file in the root of your project. This should be checked into git to manage state. 114 115 ## Excluding Files 116 117 Once the update command has been executed, the `ratchet_excluded.txt` file is created at the root of the repository. By default, this file is empty, but standard .gitignore syntax can be used to specify files that shouldn't be included in tests. Additional files that won't be tested are files specified in your gitignore and files that don't have the extension .py. 118 119 ## Running as part of PyTest 120 121 To set up tests, we provide an example file at [test_ratchet.py](https://github.com/andrewlaack/ratchets/blob/main/tests/test_files/test_ratchet.py), which defines tests to be ran with PyTest. In this file there are two uncommented methods that runs one test per rule in both sections (regex and shell). 122 123 The commented methods aggregate these tests together into two total tests (regex and shell). 124 125 When creating your PyTest file, ensure it is being indexed by PyTest. If you are unsure what this means, create a file named `test_ratchet.py` in the root of your project. 126 127 ## Running Tests 128 129 Running tests is as simple as running ```pytest``` from the root of the repository or specifying the testing file with ```pytest test_ratchet.py```. 130 131 ## Additional Functionality 132 133 Beyond a seamless integration with PyTest, Ratchets provides functionality to find the location of infringements. This and other functionality can be found by running: 134 135 ``` 136 python3 -m ratchets --help 137 ``` 138 139 Where you will see the following help message describing CLI usage for Ratchets: 140 141 ``` 142 usage: __main__.py [-h] [-t TOML_FILE] [-f FILES [FILES ...]] [-s] [-r] [-v] [-b] [--clear-cache] [-m MAX_COUNT] [-c] [-u] 143 144 Python ratchet testing 145 146 options: 147 -h, --help show this help message and exit 148 -t TOML_FILE, --toml-file TOML_FILE 149 specify a .toml file with tests 150 -f FILES [FILES ...], --files FILES [FILES ...] 151 specify file(s) to evaluate 152 -s, --shell-only run only shell-based tests 153 -r, --regex-only run only regex-based tests 154 -v, --verbose run verbose tests, printing each infringing line 155 -b, --blame run an additional git-blame for each infraction, ordering results by timestamp 156 --clear-cache clear the blame cache 157 -m MAX_COUNT, --max-count MAX_COUNT 158 maximum infractions to display per test (only applies with --blame; default is 10) 159 -c, --compare-counts show only the differences in infraction counts between the current and last saved tests 160 -u, --update-ratchets 161 update ratchets_values.json 162 ``` 163 164 **Note:** Ensure you add `.ratchet_blame.db` to your .gitignore file when using the `--blame` option. This is the location Ratchets caches blame evaluations to improve performance for larger codebases. 165 166 # Testing Ratchets Locally 167 168 To run the tests for the source code of Ratchets, you can clone this repository with: 169 170 ```bash 171 git clone https://github.com/andrewlaack/ratchets/ 172 ``` 173 174 Then `cd` into `ratchets` and run `pytest`. The tests use the installed version of Ratchets from your virtual environment. This means you must ensure changes to source files are applied to your installed `ratchets` package prior to running the tests.