testing.txt (24585B)
1 *testing.txt* For Vim version 9.2. Last change: 2026 Apr 06 2 3 4 VIM REFERENCE MANUAL by Bram Moolenaar 5 6 7 Testing Vim and Vim script *testing-support* 8 9 Expression evaluation is explained in |eval.txt|. This file goes into details 10 about writing tests in Vim script. This can be used for testing Vim itself 11 and for testing plugins. 12 13 1. Testing Vim |testing| 14 2. Test functions |test-functions-details| 15 3. Assert functions |assert-functions-details| 16 17 ============================================================================== 18 1. Testing Vim *testing* 19 20 Vim can be tested after building it, usually with "make test". 21 The tests are located in the directory "src/testdir". 22 23 There are two types of tests added over time: 24 test20.in oldest, only for tiny builds 25 test_something.vim new style tests 26 27 *new-style-testing* 28 New tests should be added as new style tests. The test scripts are named 29 test_<feature>.vim (replace <feature> with the feature under test). These use 30 functions such as |assert_equal()| to keep the test commands and the expected 31 result in one place. 32 *old-style-testing* 33 These tests are used only for testing Vim without the |+eval| feature. 34 35 Find more information in the file src/testdir/README.txt. 36 37 ============================================================================== 38 2. Test functions *test-functions-details* 39 40 test_alloc_fail({id}, {countdown}, {repeat}) *test_alloc_fail()* 41 This is for testing: If the memory allocation with {id} is 42 called, then decrement {countdown}, and when it reaches zero 43 let memory allocation fail {repeat} times. When {repeat} is 44 smaller than one it fails one time. 45 46 Can also be used as a |method|: > 47 GetAllocId()->test_alloc_fail() 48 < 49 Return type: void 50 51 52 test_autochdir() *test_autochdir()* 53 Set a flag to enable the effect of 'autochdir' before Vim 54 startup has finished. 55 56 Return type: void 57 58 59 test_feedinput({string}) *test_feedinput()* 60 Characters in {string} are queued for processing as if they 61 were typed by the user. This uses a low level input buffer. 62 This function works only when with |+unix| or GUI is running. 63 64 Can also be used as a |method|: > 65 GetText()->test_feedinput() 66 < 67 Return type: void 68 69 70 test_garbagecollect_now() *test_garbagecollect_now()* 71 Like |garbagecollect()|, but executed right away. This must 72 only be called directly to avoid any structure to exist 73 internally, and |v:testing| must have been set before calling 74 any function. *E1142* 75 This will not work when called from a :def function, because 76 variables on the stack will be freed. 77 78 Return type: void 79 80 test_garbagecollect_soon() *test_garbagecollect_soon()* 81 Set the flag to call the garbagecollector as if in the main 82 loop. Only to be used in tests. 83 84 Return type: void 85 86 test_getvalue({name}) *test_getvalue()* 87 Get the value of an internal variable. These values for 88 {name} are supported: 89 need_fileinfo 90 91 Can also be used as a |method|: > 92 GetName()->test_getvalue() 93 < 94 Return type: |Number| 95 96 *test_gui_event()* 97 test_gui_event({event}, {args}) 98 Generate a GUI {event} with arguments {args} for testing Vim 99 functionality. This function works only when the GUI is 100 running. 101 102 {event} is a String and the supported values are: 103 "dropfiles" drop one or more files in a window. 104 "findrepl" search and replace text. 105 "mouse" mouse button click event. 106 "scrollbar" move or drag the scrollbar. 107 "key" send a low-level keyboard event. 108 "tabline" select a tab page by mouse click. 109 "tabmenu" select a tabline menu entry. 110 111 {args} is a Dict and contains the arguments for the event. 112 113 "dropfiles": 114 Drop one or more files in a specified window. The supported 115 items in {args} are: 116 files: List of file names 117 row: window row number 118 col: window column number 119 modifiers: key modifiers. The supported values are: 120 0x4 Shift 121 0x8 Alt 122 0x10 Ctrl 123 The files are added to the |argument-list| and the first 124 file in {files} is edited in the window. See |drag-n-drop| 125 for more information. This event works only when the 126 |drop_file| feature is present. 127 128 "findrepl": 129 {only available when the GUI has a find/replace dialog} 130 Perform a search and replace of text. The supported items 131 in {args} are: 132 find_text: string to find. 133 repl_text: replacement string. 134 flags: flags controlling the find/replace. Supported 135 values are: 136 1 search next string (find dialog) 137 2 search next string (replace dialog) 138 3 replace string once 139 4 replace all matches 140 8 match whole words only 141 16 match case 142 forward: set to 1 for forward search. 143 144 "mouse": 145 Inject either a mouse button click, or a mouse move, event. 146 The supported items in {args} are: 147 button: mouse button. The supported values are: 148 0 left mouse button 149 1 middle mouse button 150 2 right mouse button 151 3 mouse button release 152 4 scroll wheel down 153 5 scroll wheel up 154 6 scroll wheel left 155 7 scroll wheel right 156 row: mouse click row number. The first row of the 157 Vim window is 1 and the last row is 'lines'. 158 col: mouse click column number. The maximum value 159 of {col} is 'columns'. 160 multiclick: set to 1 to inject a multiclick mouse event. 161 modifiers: key modifiers. The supported values are: 162 4 shift is pressed 163 8 alt is pressed 164 16 ctrl is pressed 165 move: Optional; if used and TRUE then a mouse move 166 event can be generated. 167 Only {args} row: and col: are used and 168 required; they are interpreted as pixels or 169 screen cells, depending on "cell". 170 Only results in an event when 'mousemoveevent' 171 is set or a popup uses mouse move events. 172 cell: Optional: when present and TRUE then "move" 173 uses screen cells instead of pixel positions 174 175 "scrollbar": 176 Set or drag the left, right or horizontal scrollbar. Only 177 works when the scrollbar actually exists. The supported 178 items in {args} are: 179 which: Selects the scrollbar. The supported values 180 are: 181 left Left scrollbar of the current window 182 right Right scrollbar of the current window 183 hor Horizontal scrollbar 184 value: Amount to scroll. For the vertical scrollbars 185 the value can be between 0 to the line-count 186 of the buffer minus one. For the horizontal 187 scrollbar the value can be between 1 and the 188 maximum line length, assuming 'wrap' is not 189 set. 190 dragging: 1 to drag the scrollbar and 0 to click in the 191 scrollbar. 192 193 "key": 194 Send a low-level keyboard event (e.g. key-up or down). 195 Currently only supported on MS-Windows. 196 The supported items in {args} are: 197 event: The supported string values are: 198 keyup generate a keyup event 199 keydown generate a keydown event 200 keycode: Keycode to use for a keyup or a keydown event. 201 *E1291* 202 203 "tabline": 204 Inject a mouse click event on the tabline to select a 205 tabpage. The supported items in {args} are: 206 tabnr: tab page number 207 208 "tabmenu": 209 Inject an event to select a tabline menu entry. The 210 supported items in {args} are: 211 tabnr: tab page number 212 item: tab page menu item number. 1 for the first 213 menu item, 2 for the second item and so on. 214 215 After injecting the GUI events you probably should call 216 |feedkeys()| to have them processed, e.g.: > 217 call feedkeys("y", 'Lx!') 218 < 219 Returns TRUE if the event is successfully added, FALSE if 220 there is a failure. 221 222 Can also be used as a |method|: > 223 GetEvent()->test_gui_event({args}) 224 < 225 Return type: |vim9-boolean| 226 227 test_ignore_error({expr}) *test_ignore_error()* 228 Ignore any error containing {expr}. A normal message is given 229 instead. 230 This is only meant to be used in tests, where catching the 231 error with try/catch cannot be used (because it skips over 232 following code). 233 {expr} is used literally, not as a pattern. 234 When the {expr} is the string "RESET" then the list of ignored 235 errors is made empty. 236 237 Can also be used as a |method|: > 238 GetErrorText()->test_ignore_error() 239 < 240 Return type: void 241 242 test_mswin_event({event}, {args}) *test_mswin_event()* 243 Generate a low-level MS-Windows {event} with arguments {args} 244 for testing Vim functionality. It works for MS-Windows GUI 245 and for the console. 246 247 {event} is a String and the supported values are: 248 "mouse" mouse event. 249 "key" keyboard event. 250 "set_keycode_trans_strategy" 251 Change the key translation method. 252 253 "mouse": 254 Inject either a mouse button click, or a mouse move, event. 255 The supported items in {args} are: 256 button: mouse button. The supported values are: 257 0 right mouse button 258 1 middle mouse button 259 2 left mouse button 260 3 mouse button release 261 4 scroll wheel down 262 5 scroll wheel up 263 6 scroll wheel left 264 7 scroll wheel right 265 row: mouse click row number. The first row of the 266 Vim window is 1 and the last row is 'lines'. 267 col: mouse click column number. The maximum value 268 of {col} is 'columns'. 269 Note: row and col are always interpreted as 270 screen cells for the console application. 271 But, they may be interpreted as pixels 272 for the GUI, depending on "cell". 273 multiclick: set to 1 to inject a double-click mouse event. 274 modifiers: key modifiers. The supported values are: 275 4 shift is pressed 276 8 alt is pressed 277 16 ctrl is pressed 278 move: Optional; if used and TRUE then a mouse move 279 event can be generated. 280 Only {args} row: and col: are used and 281 required. 282 Only results in an event when 'mousemoveevent' 283 is set or a popup uses mouse move events. 284 cell: Optional for the GUI: when present and TRUE 285 then "move" uses screen cells instead of pixel 286 positions. Not used by the console. 287 288 "key": 289 Send a low-level keyboard event (e.g. keyup or keydown). 290 The supported items in {args} are: 291 event: The supported string values are: 292 keyup generate a keyup event 293 keydown generate a keydown event 294 keycode: Keycode to use for a keyup or a keydown event. 295 modifiers: Optional; key modifiers. 296 The supported values are: 297 2 shift is pressed 298 4 ctrl is pressed 299 8 alt is pressed 300 Note: These values are different from the 301 mouse modifiers. 302 execute: Optional. Similar to |feedkeys()| mode x. 303 When this is included and set to true 304 (non-zero) then Vim will process any buffered 305 unprocessed key events. All other {args} 306 items are optional when this is set and true. 307 308 "set_keycode_trans_strategy": 309 |w32-experimental-keycode-trans-strategy| 310 Switch the keycode translation method. The supported 311 methods are: 312 experimental: The method used after Patch v8.2.4807 313 using ToUnicode() Win API call. 314 classic: The method used pre Patch v8.2.4807 315 using the TranslateMessage() Win API call. 316 317 Returns TRUE if the event is successfully added or executed, 318 FALSE if there is a failure. 319 320 Can also be used as a |method|: > 321 GetEvent()->test_mswin_event({args}) 322 < 323 Return type: |vim9-boolean| 324 325 test_null_blob() *test_null_blob()* 326 Return a |Blob| that is null. Only useful for testing. 327 328 Return type: |Blob| 329 330 test_null_channel() *test_null_channel()* 331 Return a |Channel| that is null. Only useful for testing. 332 {only available when compiled with the +channel feature} 333 334 Return type: |Channel| 335 336 test_null_dict() *test_null_dict()* 337 Return a |Dict| that is null. Only useful for testing. 338 339 Return type: dict<any> 340 341 test_null_function() *test_null_function()* 342 Return a |Funcref| that is null. Only useful for testing. 343 344 Return type: func(...): unknown 345 346 test_null_job() *test_null_job()* 347 Return a |Job| that is null. Only useful for testing. 348 {only available when compiled with the +job feature} 349 350 Return type: |job| 351 352 test_null_list() *test_null_list()* 353 Return a |List| that is null. Only useful for testing. 354 355 Return type: list<any> 356 357 test_null_partial() *test_null_partial()* 358 Return a |Partial| that is null. Only useful for testing. 359 360 Return type: func(...): unknown 361 362 test_null_string() *test_null_string()* 363 Return a |String| that is null. Only useful for testing. 364 365 Return type: |String| 366 367 test_null_tuple() *test_null_tuple()* 368 Return a |Tuple| that is null. Only useful for testing. 369 370 Return type: |Tuple| 371 372 test_option_not_set({name}) *test_option_not_set()* 373 Reset the flag that indicates option {name} was set. Thus it 374 looks like it still has the default value. Use like this: > 375 set ambiwidth=double 376 call test_option_not_set('ambiwidth') 377 < Now the 'ambiwidth' option behaves like it was never changed, 378 even though the value is "double". 379 Only to be used for testing! 380 381 Can also be used as a |method|: > 382 GetOptionName()->test_option_not_set() 383 < 384 Return type: void 385 386 test_override({name}, {val}) *test_override()* 387 Overrides certain parts of Vim's internal processing to be 388 able to run tests. Only to be used for testing Vim! 389 The override is enabled when {val} is non-zero and removed 390 when {val} is zero. 391 Current supported values for {name} are: 392 393 {name} effect when {val} is non-zero ~ 394 alloc_lines make a copy of every buffer line into allocated 395 memory, so that memory access errors can be found 396 by valgrind. 397 autoload `import autoload` will load the script right 398 away, not postponed until an item is used. 399 char_avail disable the char_avail() function. 400 defcompile all the |:def| functions in a sourced script are 401 compiled when defined. This is similar to using 402 the |:defcompile| command in a script. 403 nfa_fail makes the NFA regexp engine fail to force a 404 fallback to the old engine. 405 no_query_mouse do not query the mouse position for "dec" 406 terminals. 407 no_wait_return set the "no_wait_return" flag. Not restored 408 with "ALL". 409 redraw disable the redrawing() function. 410 redraw_flag ignore the RedrawingDisabled flag. 411 starting reset the "starting" variable, see below. 412 syn_idlist_cache disable the cache for resolving 413 syntax "contains"/cluster lists. 414 term_props reset all terminal properties when the version 415 string is detected. 416 ui_delay time in msec to use in ui_delay(); overrules a 417 wait time of up to 3 seconds for messages. 418 unreachable no error for code after `:throw` and `:return`. 419 uptime overrules sysinfo.uptime. 420 vterm_title setting the window title by a job running in a 421 terminal window. 422 ALL clear all overrides, except alloc_lines ({val} is 423 not used). 424 425 "starting" is to be used when a test should behave like 426 startup was done. Since the tests are run by sourcing a 427 script the "starting" variable is non-zero. This is usually a 428 good thing (tests run faster), but sometimes this changes 429 behavior in a way that the test doesn't work properly. 430 When using: > 431 call test_override('starting', 1) 432 < The value of "starting" is saved. It is restored by: > 433 call test_override('starting', 0) 434 435 < To make sure the flag is reset later using `:defer` can be 436 useful: > 437 call test_override('unreachable', 1) 438 defer call test_override('unreachable', 0) 439 440 < Can also be used as a |method|: > 441 GetOverrideVal()-> test_override('starting') 442 < 443 Return type: void 444 445 test_refcount({expr}) *test_refcount()* 446 Return the reference count of {expr}. When {expr} is of a 447 type that does not have a reference count, returns -1. Only 448 to be used for testing. 449 450 Can also be used as a |method|: > 451 GetVarname()->test_refcount() 452 < 453 Return type: |Number| 454 455 test_setmouse({row}, {col}) *test_setmouse()* 456 Set the mouse position to be used for the next mouse action. 457 {row} and {col} are one based. 458 For example: > 459 call test_setmouse(4, 20) 460 call feedkeys("\<LeftMouse>", "xt") 461 < 462 Return type: void 463 464 465 test_settime({expr}) *test_settime()* 466 Set the time Vim uses internally. Currently only used for 467 timestamps in the history, as they are used in viminfo, and 468 for undo. 469 Using a value of 1 makes Vim not sleep after a warning or 470 error message. 471 {expr} must evaluate to a number. When the value is zero the 472 normal behavior is restored. 473 474 Can also be used as a |method|: > 475 GetTime()->test_settime() 476 < 477 Return type: void 478 479 test_srand_seed([{seed}]) *test_srand_seed()* 480 When {seed} is given this sets the seed value used by 481 `srand()`. When omitted the test seed is removed. 482 483 Return type: void 484 485 test_unknown() *test_unknown()* 486 Return a value with unknown type. Only useful for testing. 487 488 Return type: unknown 489 490 test_void() *test_void()* 491 Return a value with void type. Only useful for testing. 492 493 Return type: void 494 495 ============================================================================== 496 3. Assert functions *assert-functions-details* 497 498 499 assert_beeps({cmd}) *assert_beeps()* 500 Run {cmd} and add an error message to |v:errors| if it does 501 NOT produce a beep or visual bell. 502 Also see |assert_fails()|, |assert_nobeep()| and 503 |assert-return|. 504 505 Can also be used as a |method|: > 506 GetCmd()->assert_beeps() 507 < 508 Return type: |Number| 509 510 *assert_equal()* 511 assert_equal({expected}, {actual} [, {msg}]) 512 When {expected} and {actual} are not equal an error message is 513 added to |v:errors| and 1 is returned. Otherwise zero is 514 returned. |assert-return| 515 The error is in the form "Expected {expected} but got 516 {actual}". When {msg} is present it is prefixed to that, 517 along with the location of the assert when run from a script. 518 519 There is no automatic conversion, the String "4" is different 520 from the Number 4. And the number 4 is different from the 521 Float 4.0. The value of 'ignorecase' is not used here, case 522 always matters. 523 Example: > 524 call assert_equal('foo', 'bar', 'baz') 525 < Will add the following to |v:errors|: 526 test.vim line 12: baz: Expected 'foo' but got 'bar' ~ 527 528 Can also be used as a |method|, the base is passed as the 529 second argument: > 530 mylist->assert_equal([1, 2, 3]) 531 < 532 Return type: |Number| 533 534 *assert_equalfile()* 535 assert_equalfile({fname-one}, {fname-two} [, {msg}]) 536 When the files {fname-one} and {fname-two} do not contain 537 exactly the same text an error message is added to |v:errors|. 538 Also see |assert-return|. 539 When {fname-one} or {fname-two} does not exist the error will 540 mention that. 541 Mainly useful with |terminal-diff|. 542 543 Can also be used as a |method|: > 544 GetLog()->assert_equalfile('expected.log') 545 < 546 Return type: |Number| 547 548 assert_exception({error} [, {msg}]) *assert_exception()* 549 When v:exception does not contain the string {error} an error 550 message is added to |v:errors|. Also see |assert-return|. 551 This can be used to assert that a command throws an exception. 552 Using the error number, followed by a colon, avoids problems 553 with translations: > 554 try 555 commandthatfails 556 call assert_false(1, 'command should have failed') 557 catch 558 call assert_exception('E492:') 559 endtry 560 < 561 Return type: |Number| 562 563 *assert_fails()* 564 assert_fails({cmd} [, {error} [, {msg} [, {lnum} [, {context}]]]]) 565 Run {cmd} and add an error message to |v:errors| if it does 566 NOT produce an error or when {error} is not found in the 567 error message. Also see |assert-return|. 568 *E856* 569 When {error} is a string it must be found literally in the 570 first reported error. Most often this will be the error code, 571 including the colon, e.g. "E123:". > 572 call assert_fails('bad cmd', 'E987:') 573 < 574 When {error} is a |List| with one or two strings, these are 575 used as patterns. The first pattern is matched against the 576 first reported error: > 577 call assert_fails('cmd', ['E987:.*expected bool']) 578 < The second pattern, if present, is matched against the last 579 reported error. 580 If there is only one error then both patterns must match. 581 This can be used to check that there is only one error. 582 To only match the last error use an empty string for the first 583 error: > 584 call assert_fails('cmd', ['', 'E987:']) 585 < 586 If {msg} is empty then it is not used. Do this to get the 587 default message when passing the {lnum} argument. 588 *E1115* 589 When {lnum} is present and not negative, and the {error} 590 argument is present and matches, then this is compared with 591 the line number at which the error was reported. That can be 592 the line number in a function or in a script. 593 *E1116* 594 When {context} is present it is used as a pattern and matched 595 against the context (script name or function name) where 596 {lnum} is located in. 597 598 Note that beeping is not considered an error, and some failing 599 commands only beep. Use |assert_beeps()| for those. 600 601 Can also be used as a |method|: > 602 GetCmd()->assert_fails('E99:') 603 < 604 Return type: |Number| 605 606 assert_false({actual} [, {msg}]) *assert_false()* 607 When {actual} is not false an error message is added to 608 |v:errors|, like with |assert_equal()|. 609 The error is in the form "Expected False but got {actual}". 610 When {msg} is present it is prefixed to that, along with the 611 location of the assert when run from a script. 612 Also see |assert-return|. 613 614 A value is false when it is zero. When {actual} is not a 615 number the assert fails. 616 617 Can also be used as a |method|: > 618 GetResult()->assert_false() 619 < 620 Return type: |Number| 621 622 assert_inrange({lower}, {upper}, {actual} [, {msg}]) *assert_inrange()* 623 This asserts number and |Float| values. When {actual} is lower 624 than {lower} or higher than {upper} an error message is added 625 to |v:errors|. Also see |assert-return|. 626 The error is in the form "Expected range {lower} - {upper}, 627 but got {actual}". When {msg} is present it is prefixed to 628 that. 629 630 Return type: |Number| 631 632 *assert_match()* 633 assert_match({pattern}, {actual} [, {msg}]) 634 When {pattern} does not match {actual} an error message is 635 added to |v:errors|. Also see |assert-return|. 636 The error is in the form "Pattern {pattern} does not match 637 {actual}". When {msg} is present it is prefixed to that, 638 along with the location of the assert when run from a script. 639 640 {pattern} is used as with |=~|: The matching is always done 641 like 'magic' was set and 'cpoptions' is empty, no matter what 642 the actual value of 'magic' or 'cpoptions' is. 643 644 {actual} is used as a string, automatic conversion applies. 645 Use "^" and "$" to match with the start and end of the text. 646 Use both to match the whole text. 647 648 Example: > 649 call assert_match('^f.*o$', 'foobar') 650 < Will result in a string to be added to |v:errors|: 651 test.vim line 12: Pattern '^f.*o$' does not match 'foobar' ~ 652 653 Can also be used as a |method|: > 654 getFile()->assert_match('foo.*') 655 < 656 Return type: |Number| 657 658 assert_nobeep({cmd}) *assert_nobeep()* 659 Run {cmd} and add an error message to |v:errors| if it 660 produces a beep or visual bell. 661 Also see |assert_beeps()|. 662 663 Can also be used as a |method|: > 664 GetCmd()->assert_nobeep() 665 < 666 Return type: |Number| 667 668 *assert_notequal()* 669 assert_notequal({expected}, {actual} [, {msg}]) 670 The opposite of `assert_equal()`: add an error message to 671 |v:errors| when {expected} and {actual} are equal. 672 Also see |assert-return|. 673 674 Can also be used as a |method|: > 675 mylist->assert_notequal([1, 2, 3]) 676 < 677 Return type: |Number| 678 679 *assert_notmatch()* 680 assert_notmatch({pattern}, {actual} [, {msg}]) 681 The opposite of `assert_match()`: add an error message to 682 |v:errors| when {pattern} matches {actual}. 683 Also see |assert-return|. 684 685 Can also be used as a |method|: > 686 getFile()->assert_notmatch('bar.*') 687 < 688 Return type: |Number| 689 690 assert_report({msg}) *assert_report()* 691 Report a test failure directly, using String {msg}. 692 Always returns one. 693 694 Can also be used as a |method|: > 695 GetMessage()->assert_report() 696 < 697 Return type: |Number| 698 699 700 assert_true({actual} [, {msg}]) *assert_true()* 701 When {actual} is not true an error message is added to 702 |v:errors|, like with |assert_equal()|. 703 Also see |assert-return|. 704 A value is TRUE when it is a non-zero number. When {actual} 705 is not a number the assert fails. 706 When {msg} is given it is prefixed to the default message, 707 along with the location of the assert when run from a script. 708 709 Can also be used as a |method|: > 710 GetResult()->assert_true() 711 < 712 Return type: |Number| 713 714 715 vim:tw=78:ts=8:noet:ft=help:norl: