blog

Personal blog
git clone git://git.laack.co/blog.git
Log | Files | Refs

to-make-a-cool-background.md (26471B)


      1 # To Make a Cool Background
      2 
      3 I wrote a visualization program for prim's algorithm that induces a minimum spanning tree (MST) on a procedurally generated graph. How does one go about making this their background and screensaver with X11 / dwm?
      4 
      5 ## MST
      6 
      7 IMPLEMENTATION DETAILS
      8 
      9 ## Background making
     10 
     11 [https://github.com/python-xlib/python-xlib](https://github.com/python-xlib/python-xlib.md)
     12 
     13 ```python3
     14 os.environ["SDL_VIDEODRIVER"] = "dummy"
     15 ```
     16 
     17 why not just use screenshots in a shared location? 
     18 
     19 indeed, why not?
     20 
     21 ## fuck you and your bullshit.
     22 
     23 Benchmarking
     24 
     25 ## c -> c++
     26 
     27 realized I needed a heap.
     28 
     29 ## bmp vs png vs jpg
     30 
     31 bmp is crazy fucking fast (benchmark)
     32 
     33 (looked in the code for supported file formats, looked in feh -> lib(whatever) for support)
     34 
     35 note larger file size.
     36 
     37 this is interesting, look further.
     38 
     39 ### patched upstream xl
     40 
     41 ### x11
     42 
     43 x11 is weird with this because they are kind hacky in that they just put up a window and take away your inputs, forcing them to go to that window.
     44 
     45 (should learn more about this)
     46 
     47 ### Benchmarking
     48 
     49 uhhh... is it actually that fast?
     50 
     51 wait; did this just optimize all of my code?
     52 
     53 Well yes, but actually no. 
     54 
     55 adding:
     56 
     57 std::cout << g.toString() << std::endl;
     58 
     59 gives ~ the same perf (minus the amount of time it takes to send so much data to stdout)
     60 
     61 (this would require the full traversal because toString is dependent upon prior prim algo steps being performed otherwise the traversal status for edges and vertices would be incorrect).
     62 
     63 (fixed a bug in the python code slowing it down)
     64 
     65 see no_render_2_fixed_itr.
     66 
     67 ## C++ benchmarking
     68 
     69 330 ms for init stuff w/ graph
     70 1405 ms for iteration
     71 
     72 broken further down:
     73 
     74 average iteration time is ~0ms, reaching ~20ms for later stages. Over time the percentage of valid edges declines.
     75 
     76 added getUnvisitedEdges function to return only non-traversed edges
     77 
     78 Init time: 332
     79 Loop time: 1344
     80 Init time: 327
     81 Loop time: 1350
     82 Init time: 322
     83 Loop time: 1374
     84 
     85 still has a similar issue as before; degredation in perf towards the end bc things are pushed earlier on and then get invalidated later on.
     86 
     87 that's basically all I could squeeze out without changes to the algo to be eager instead of lazy wrt tracking vertices. Basically, what's the shortest distance to a given vertex instead of what's the edge that we have that's shortest. 
     88 
     89 such a refactor though is likely irrelevant right now due to the added cost of 
     90 
     91 ope; realized my function for get edges with unvisited vertices above was wrong. I was just not pushing visited edges, but we really care about vertices that haven't been visited so refactored the edge generation and stuff to get a stable ref to v2index and then push only if that hasn't been visited.
     92 
     93 Init time: 389
     94 Loop time: 706
     95 Init time: 315
     96 Loop time: 671
     97 Init time: 337
     98 Loop time: 692
     99 
    100 now we're cooking with gas.
    101 
    102 tests now fail. bc snapshots. updated and everything seems right again.
    103 
    104 actually the stable things isn't really necessary as this approach that's unstable is functionally the same, but safer in case there are changes to the IR of edges.
    105 
    106 Init time: 334
    107 Loop time: 694
    108 Init time: 321
    109 Loop time: 668
    110 Init time: 323
    111 Loop time: 701
    112 
    113 now we find this:
    114 
    115 (see benchmarking/no_render_v_100000_e_1000000_s_0/out1.csv)
    116 
    117 same optimization performed for python code so that's then down to 8.6 seconds ~~~ see andrew@deepthought:no_render_3_fixed_itr_fixed_check$
    118 
    119 at this point it's worth worrying about the return for the get untraversed edges for vertices not traversed. after that:
    120 
    121 Init time: 314
    122 Loop time: 678
    123 Init time: 312
    124 Loop time: 671
    125 Init time: 311
    126 Loop time: 680
    127 
    128 and with the optimization for edge ordering:
    129 
    130 Init time: 311
    131 Loop time: 659
    132 Init time: 333
    133 Loop time: 673
    134 Init time: 320
    135 Loop time: 709
    136 Init time: 311
    137 Loop time: 674
    138 
    139 this is still too fucking slow.
    140 
    141 what's making the graph induction so incredibly slow?
    142 
    143 just graph creation:
    144 
    145 graph create: 317
    146 Loop time: 691
    147 graph create: 314
    148 Loop time: 657
    149 graph create: 311
    150 Loop time: 665
    151 
    152 reserving instead of pushing:
    153 
    154 graph create: 314
    155 Loop time: 636
    156 graph create: 324
    157 Loop time: 688
    158 graph create: 315
    159 Loop time: 693
    160 
    161 replaced map with vector where we were using indices to index into the map (dumb):
    162 
    163 graph create: 174
    164 Loop time: 654
    165 graph create: 179
    166 Loop time: 610
    167 graph create: 184
    168 Loop time: 618
    169 graph create: 185
    170 Loop time: 622
    171 
    172 ok, let's now run the full benchmarks again:
    173 
    174 benchmarking/no_render_only_grab_unvisited_edges_vector_instead_of_map_v_100000_e_1000000_s_0/out1.csv
    175 
    176 0.04,0.79,0.84
    177 0.03,0.79,0.84
    178 0.04,0.77,0.82
    179 
    180 this was kind of just stalling though becaus I'm out of my depth as it relates to rendering, the larger perf issue.
    181 
    182 back to that. 
    183 
    184 here are where this stands w/ 1000 v and 10_000 edges
    185 
    186 prior was ~34s per iteration (final col)
    187 
    188 this is basically the same:
    189 
    190 benchmarking/better_render_v_1000_e_10000_s_0/out1.csv
    191 
    192 (snippet from above file)
    193 
    194 0.26,10.71,34.22
    195 0.29,10.64,33.24
    196 0.31,10.62,33.57
    197 0.28,10.53,33.56
    198 
    199 fine. we'll fix the raylib shit. (queue montage)
    200 
    201 make debug-build && time ./abg.out -s 0 --edges 2000 --vertices 200
    202 
    203 real    0m1.948s
    204 user    0m0.529s
    205 sys     0m0.060s
    206 
    207 ~20 milliseconds per render call w/ make debug-build && time ./abg.out -s 0 --edges 20000 --vertices 2000
    208 
    209 Total single iteration cost: 65
    210 Amount of time calling render: 18
    211 Total single iteration cost: 65
    212 Amount of time calling render: 18
    213 Total single iteration cost: 65
    214 Amount of time calling render: 18
    215 Total single iteration cost: 65
    216 Amount of time calling render: 18
    217 Total single iteration cost: 65
    218 Amount of time calling render: 19
    219 Total single iteration cost: 65
    220 Amount of time calling render: 18
    221 Total single iteration cost: 65
    222 Amount of time calling render: 18
    223 Total single iteration cost: 65
    224 
    225 huh; what's the other iteration cost?
    226 
    227 
    228     while (!WindowShouldClose() && toVisit.size() != 0) {
    229         auto startLoop = std::chrono::steady_clock::now();
    230         BeginDrawing();
    231         ClearBackground(BLACK);
    232         auto startRender = std::chrono::steady_clock::now();
    233         g.render();
    234         auto endRender = std::chrono::steady_clock::now();
    235         auto diff = std::chrono::duration_cast<std::chrono::milliseconds>(endRender - startRender);
    236         std::cout << "Amount of time calling render: " << diff.count() << std::endl;
    237         EndDrawing();
    238 
    239         // since we wait sleepTime here, the bg render has a render delta of
    240         // at minimum sleepTime when switching tags in dwm, this is rather
    241         // annoying because the screen doesn't repaint until the sleep time
    242         // passes, which results in artifacts on screen.
    243 
    244         // despite this, calling render a lot of times is rather intensive
    245         // (at least on my hardware) and so this tradeoff is accepted for
    246         // now, unless there's a simple approach that allows for preemption
    247 
    248         //usleep((int)(sleepTime * 1000000));
    249 
    250         oneStepPrim(toVisit, visitedIndices, g);
    251         auto endLoop = std::chrono::steady_clock::now();
    252         auto loopDiff = std::chrono::duration_cast<std::chrono::milliseconds>(endLoop - startLoop);
    253         std::cout << "Total single iteration cost: " << loopDiff.count() << std::endl;
    254 
    255     }
    256 
    257 so it's somewhere in that first block; in the render area. the actual computation cost at the end is inconsequential. 
    258 
    259 begindrawing is ~free.
    260 
    261 clear background is ~free.
    262 
    263 ~46 of 65ms are spent on enddrawing. 
    264 
    265 so two things:
    266 
    267 19ms on render
    268 46ms on enddrawing
    269 
    270 what about when the numbers are much bigger (100_000 edges, 10_000 vertices)?
    271 
    272 Render time: 56
    273 End Drawing time: 237
    274 Total single iteration cost: 316
    275 
    276 the other 23 ms is the prim step + begin drawing / clear background.
    277 
    278 can we just not clear the screen? does that help? nope. doesn't do anything.
    279 
    280 target 60 fps?
    281 
    282 nope.
    283 
    284 ahh; the slowness is being realized later on. while render is cool with me double calling on edges; this is felt when enddrawing is called as the gpu does the rendering and stuff at that point. 
    285 
    286 what happens if I use the better single time render method?
    287 
    288 before:
    289 
    290 Render time: 62
    291 End Drawing time: 244
    292 Total single iteration cost: 306
    293 Render time: 60
    294 End Drawing time: 245
    295 Total single iteration cost: 305
    296 Render time: 60
    297 End Drawing time: 245
    298 Total single iteration cost: 305
    299 Render time: 55
    300 End Drawing time: 252
    301 Total single iteration cost: 307
    302 Render time: 60
    303 End Drawing time: 245
    304 Total single iteration cost: 305
    305 Render time: 58
    306 End Drawing time: 247
    307 Total single iteration cost: 306
    308 
    309 after:
    310 
    311 Total single iteration cost: 158
    312 Render time: 46
    313 End Drawing time: 112
    314 Total single iteration cost: 159
    315 Render time: 46
    316 End Drawing time: 112
    317 Total single iteration cost: 159
    318 Render time: 45
    319 End Drawing time: 113
    320 Total single iteration cost: 159
    321 Render time: 46
    322 End Drawing time: 111
    323 Total single iteration cost: 158
    324 Render time: 46
    325 End Drawing time: 112
    326 Total single iteration cost: 158
    327 Render time: 46
    328 End Drawing time: 113
    329 
    330 hell yeah.
    331 
    332 why so slow still?
    333 
    334 Like this is only ~100_000 and ~10_000 vertices so like why so slow?
    335 
    336 benchmarked now:
    337 
    338 benchmarking/simpler_render_v_1000_e_10000_s_0/out1.csv
    339 0.29,7.77,18.96
    340 0.27,7.80,19.17
    341 0.29,7.90,19.17
    342 0.31,7.76,19.05
    343 0.29,7.72,18.68
    344 0.30,7.90,18.56
    345 0.27,7.78,18.80
    346 0.27,7.77,18.74
    347 0.29,7.69,18.77
    348 0.32,7.67,18.85
    349 
    350 alright then. unfortunately, we're only calling the draw function once per vertex and edge now. we could change the rendering to only render connections, this would increase speed a lot, but it also makes the animation look less cool so fuck that. 
    351 
    352 could we draw in the background? like create a new thread that doesn't block us and then wait until that's done once we get back?
    353 
    354 well maybe, but the issue is I don't know how well this handles drawing while there's a queue of messages still being sent. 
    355 
    356 std::thread t(endDrawing);
    357 t.join();
    358 
    359 vs endDrawing()
    360 
    361 result in entirely different outcomes.
    362 
    363 the second sort of renders some stuff, but gets totally fucked up and then just black screens.
    364 
    365 due to an import? no this shit's just fucked.
    366 
    367 opengl affinity.
    368 
    369 can prebake the graph though and then only render additions.
    370 
    371 after:
    372 
    373 0.19,1.77,3.86
    374 0.17,1.76,3.81
    375 0.19,1.76,3.82
    376 0.18,1.75,3.85
    377 0.18,1.71,3.86
    378 0.19,1.68,3.83
    379 0.18,1.72,3.83
    380 
    381 holy shit. can we make this faster?
    382 
    383 of fucking course we can. track if a traversed state has been rendered and skip it if it has, using this to update our incremental texture. we then render that. 
    384 
    385 benchmarking/cache_first_use_blank_graph_v_1000_e_10000_s_0/out1.csv
    386 
    387 0.17,0.65,3.06
    388 0.18,0.63,3.05
    389 0.18,0.66,3.05
    390 0.18,0.64,3.05
    391 0.18,0.64,3.04
    392 0.17,0.63,3.02
    393 0.18,0.62,3.04
    394 0.19,0.64,3.05
    395 0.17,0.65,3.04
    396 0.17,0.66,3.03
    397 0.18,0.62,3.04
    398 0.17,0.63,3.03
    399 
    400 this is too boring; let's turn this shit up.
    401 
    402 how fast are we relative to the python bloatware webshit?
    403 
    404 benchmarking/cache_first_use_blank_graph_v_10000_e_100000_s_0/out1.csv
    405 
    406 1.40,28.92,35.76
    407 1.53,29.30,35.95
    408 
    409 (notice the edges and vertices)
    410 
    411 vs python:
    412 
    413 1.33,10730.71,10772.74
    414 
    415 that's only >300x slower for the python code so the average python developer would totally be cool with shipping it. Job done, webshit: built. 
    416 
    417 alright then. enough messing around. We really should have a queue for items that are to be rendered so we don't have to iterate over all vertices and edges, checking if they have the 'rendered' flag set. 
    418 
    419 after doing this we find:
    420 
    421 benchmarking/track_to_render_v_10000_e_100000_s_0/out1.csv
    422 
    423 1.50,3.12,29.43
    424 1.51,3.20,29.67
    425 1.57,3.08,30.00
    426 1.57,3.04,30.07
    427 
    428 35.855000000000004 / 29.7925 = 1.2034908114458338
    429 
    430 okay, so 20 percent faster. still too fucking slow.
    431 
    432 i'm also getting sick of this bs chrono / time shit. time to bring in perf.
    433 
    434  3.20%           395  abg.out  libraylib.so.6.0.0            [.] rlVertex3f
    435  2.08%           338  abg.out  abg.out                       [.] Edge::operator>(Edge const&) const
    436  1.55%           245  abg.out  abg.out                       [.] oneStepPrim(std::priority_queue<Edge, std::vector<Edge, st>
    437  1.41%           255  abg.out  libraylib.so.6.0.0            [.] rlDrawRenderBatch
    438  1.22%           225  abg.out  libc.so.6                     [.] 0x0000000000185dde
    439  0.85%           129  abg.out  libraylib.so.6.0.0            [.] DrawCircleSector
    440  0.79%           136  abg.out  libc.so.6                     [.] pthread_mutex_lock
    441  0.74%           133  abg.out  libc.so.6                     [.] ioctl
    442  0.64%           115  abg.out  libgallium-26.2.2-arch1.1.so  [.] 0x00000000007f8138
    443  0.62%           113  abg.out  libraylib.so.6.0.0            [.] PollInputEvents
    444  0.57%            91  abg.out  libc.so.6                     [.] cfree
    445  0.56%            95  abg.out  libc.so.6                     [.] malloc
    446  0.54%            27  abg.out  libc.so.6                     [.] 0x00000000001866c9
    447  0.53%            99  abg.out  libgallium-26.2.2-arch1.1.so  [.] 0x000000000180c45c
    448 
    449 alright so raylib is spending a decent amount of time on Vertex3f though not a crazy amount. Comparisons between edges are common; not surprising as we are using a heap with lots of elements towards the end, as we've seen. prim's algorithm generally taking some time, makes sense.
    450 
    451 render drawing is rather low, but within reason. This is rather disappointing. It's nice when it's like, no, dude, you spent 99.95% of your time in one function. 
    452 
    453 looking at the call stack:
    454 
    455 |--14.80%--EndTextureMode
    456 |--58.02%--EndDrawing
    457 --1.62%--Graph::render()
    458 
    459 well. huh. end texture mode is where I add things though the function itself is just clearing the queue. 
    460 
    461 It's a bit surprising enddrawing is more expensive since it just loads in the texture I've baked. 
    462 
    463 ---
    464 
    465 I shall call this ~done. final findings with 
    466 
    467 rendered below is using     vertices = 10_000, edges = 100_000
    468 
    469 unrendered below is using   vertices = 200_000, edges = 2_000_000 
    470 
    471 - python (with overdraw mostly fixing + texture map)
    472     - rendered 
    473         - 58.758 seconds
    474     - unrendered
    475         - 20.253333333333334 seconds
    476 
    477 - c++ (with overdraw fixing + texture map)
    478     - rendered
    479         - - 29.7925 seconds
    480 
    481     - unrendered
    482         - 2.118 seconds
    483 
    484 
    485 benchmarking/final_no_render_v_200000_e_2000000_s_0/
    486 
    487 takeaway:
    488 
    489 the rendered python implementation w/ 10_000 v and 100_000 edges using perf:
    490 
    491 Performance counter stats for 'python3 prim.py':
    492 
    493                  0      context-switches:u               #      0.0 cs/sec  cs_per_second
    494                  0      cpu-migrations:u                 #      0.0 migrations/sec  migrations_per_second
    495             14,233      page-faults:u                    #    252.0 faults/sec  page_faults_per_second
    496          56,487.57 msec task-clock:u                     #      nan CPUs  CPUs_utilized
    497         30,763,483      branch-misses:u                  #      0.7 %  branch_miss_rate         (88.90%)
    498      4,305,442,939      branches:u                       #     76.2 M/sec  branch_frequency     (88.90%)
    499    128,137,043,750      cpu-cycles:u                     #      2.3 GHz  cycles_frequency       (88.90%)
    500     18,736,054,409      instructions:u                   #      0.1 instructions  insn_per_cycle  (88.85%)
    501             TopdownL1 #      0.1 %  tma_backend_bound
    502                                                          #     98.9 %  tma_bad_speculation      (88.87%)
    503                                                          #      0.5 %  tma_frontend_bound       (77.81%)
    504                                                          #      0.5 %  tma_retiring             (88.92%)
    505 
    506       59.936596362 seconds time elapsed
    507 
    508       52.628026000 seconds user
    509        1.492521000 seconds sys
    510 
    511 gpu constrained though. 
    512 
    513 the rendered c++ implementation w/ 10_000 v and 100_000 edges using perf:
    514 
    515  Performance counter stats for './abg.out -s 0 --vertices 10000 --edges 100000':
    516 
    517                  0      context-switches:u               #      0.0 cs/sec  cs_per_second
    518                  0      cpu-migrations:u                 #      0.0 migrations/sec  migrations_per_second
    519              9,363      page-faults:u                    #   1819.4 faults/sec  page_faults_per_second
    520           5,146.21 msec task-clock:u                     #      nan CPUs  CPUs_utilized
    521         28,144,835      branch-misses:u                  #      5.9 %  branch_miss_rate         (89.55%)
    522        473,319,266      branches:u                       #     92.0 M/sec  branch_frequency     (88.64%)
    523      3,973,233,282      cpu-cycles:u                     #      0.8 GHz  cycles_frequency       (88.55%)
    524      2,925,854,560      instructions:u                   #      0.7 instructions  insn_per_cycle  (89.12%)
    525             TopdownL1 #      2.7 %  tma_backend_bound
    526                                                          #     93.6 %  tma_bad_speculation      (88.66%)
    527                                                          #      0.9 %  tma_frontend_bound       (77.75%)
    528                                                          #      2.8 %  tma_retiring             (88.67%)
    529 
    530       29.577542431 seconds time elapsed
    531 
    532        2.925597000 seconds user
    533        2.094917000 seconds sys
    534 
    535 32x less instructions than the python version. Neither is cpu constrained, but this is a non-trivial amount of overhead from the python version; c++ version also had 7x higher instructions per cycle. 
    536 
    537 gpu constrained too.
    538 
    539 memory overhead?
    540 
    541 c++ rendered sits at ~171mb-175mb. it's not great, hasn't been optimized, def room for improvement. 
    542 
    543 python rendered sits at ~240mb-248mb
    544 
    545 the default invocation method for the c++ implementation I use as my lock screen is running at 160mb. This is rather annoying, but also, it's doing a bunch of rendering stuff so IG that's fine, and this does improve the computational cost in terms of cycles so...
    546 
    547 huh... after 2 hours it's chilling at 163mb (TODO).
    548 
    549 bc it's always holding onto at least one frame which is 5120x1440 pixels which is ~21mb minimum (assuming r g and b bytes per pixel.)
    550 
    551 an eager approach for this would be better, but c++ stl doesn't have an indexed priority queue, so I'll just retcon what I have. 
    552 
    553 basically, I just want to minimize the useless things I push to the queue. One way to do this is to track the minimum weighted edge with an untraversed vertex and then updating this and only pushing edges with it when they are < that weight. 
    554 
    555 before:
    556 
    557 benchmarking/final_no_render_v_200000_e_2000000_s_0/
    558 
    559 0.08,2.03,2.12
    560 0.09,2.02,2.12
    561 0.09,2.00,2.11
    562 0.09,2.00,2.11
    563 0.08,1.99,2.09
    564 0.07,1.99,2.08
    565 0.10,2.00,2.11
    566 0.09,2.00,2.11
    567 0.09,2.07,2.18
    568 0.06,2.08,2.15
    569 
    570 
    571 after:
    572 
    573 benchmarking/final_min_no_render_v_200000_e_2000000_s_0/out.csv
    574 
    575 0.06,0.97,1.04
    576 0.06,0.93,1.00
    577 0.07,0.95,1.03
    578 0.09,0.92,1.02
    579 0.08,0.94,1.02
    580 0.08,0.94,1.03
    581 0.08,0.94,1.03
    582 0.06,0.95,1.02
    583 0.08,0.93,1.02
    584 0.07,0.95,1.02
    585 0.07,0.95,1.03
    586 0.07,0.95,1.03
    587 0.08,0.95,1.04
    588 0.09,0.93,1.03
    589 0.07,0.98,1.06
    590 0.07,0.95,1.04
    591 0.08,0.94,1.03
    592 0.07,0.96,1.04
    593 0.07,0.95,1.04
    594 0.07,1.00,1.08
    595 
    596 this doesn't really impact time wrt rendered because that is basically all spent on rendering not computation. 
    597 
    598 do I have a memory leak?
    599 
    600 
    601 
    602 > time valgrind --tool=memcheck abg
    603 
    604 ==944657== Process terminating with default action of signal 2 (SIGINT)
    605 ==944657==    at 0x50D3952: __syscall_cancel_arch (syscall_cancel.S:56)
    606 ==944657==    by 0x5117AEC: internal_syscall_cancel (sysdep-cancel.h:53)
    607 ==944657==    by 0x5117AEC: clock_nanosleep@@GLIBC_2.17 (clock_nanosleep.c:48)
    608 ==944657==    by 0x5123F26: nanosleep (nanosleep.c:25)
    609 ==944657==    by 0x51532E9: usleep (usleep.c:31)
    610 ==944657==    by 0x40065B8: main (in /usr/local/bin/abg)
    611 ==944657==
    612 ==944657== HEAP SUMMARY:
    613 ==944657==     in use at exit: 11,776,171 bytes in 27,368 blocks
    614 ==944657==   total heap usage: 86,922 allocs, 59,554 frees, 29,848,973 bytes allocated
    615 ==944657==
    616 ==944657== LEAK SUMMARY:
    617 ==944657==    definitely lost: 0 bytes in 0 blocks
    618 ==944657==    indirectly lost: 0 bytes in 0 blocks
    619 ==944657==      possibly lost: 6,876,540 bytes in 3,237 blocks
    620 ==944657==    still reachable: 4,899,631 bytes in 24,131 blocks
    621 ==944657==         suppressed: 0 bytes in 0 blocks
    622 ==944657== Rerun with --leak-check=full to see details of leaked memory
    623 ==944657==
    624 ==944657== For lists of detected and suppressed errors, rerun with: -s
    625 ==944657== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
    626 
    627 
    628 real    12m18.560s
    629 user    0m23.848s
    630 sys     0m0.969s
    631 
    632 andrew@deepthought:~$ time valgrind --tool=memcheck abg
    633 ==947393== Memcheck, a memory error detector
    634 ==947393== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
    635 ==947393== Using Valgrind-3.25.1 and LibVEX; rerun with -h for copyright info
    636 ==947393== Command: abg
    637 ==947393==
    638 ^C==947393==
    639 ==947393== Process terminating with default action of signal 2 (SIGINT)
    640 ==947393==    at 0x50D3952: __syscall_cancel_arch (syscall_cancel.S:56)
    641 ==947393==    by 0x5117AEC: internal_syscall_cancel (sysdep-cancel.h:53)
    642 ==947393==    by 0x5117AEC: clock_nanosleep@@GLIBC_2.17 (clock_nanosleep.c:48)
    643 ==947393==    by 0x5123F26: nanosleep (nanosleep.c:25)
    644 ==947393==    by 0x51532E9: usleep (usleep.c:31)
    645 ==947393==    by 0x40065B8: main (in /usr/local/bin/abg)
    646 ==947393==
    647 ==947393== HEAP SUMMARY:
    648 ==947393==     in use at exit: 11,656,499 bytes in 27,404 blocks
    649 ==947393==   total heap usage: 64,820 allocs, 37,416 frees, 24,650,054 bytes allocated
    650 ==947393==
    651 ==947393== LEAK SUMMARY:
    652 ==947393==    definitely lost: 0 bytes in 0 blocks
    653 ==947393==    indirectly lost: 0 bytes in 0 blocks
    654 ==947393==      possibly lost: 6,744,916 bytes in 3,235 blocks
    655 ==947393==    still reachable: 4,911,583 bytes in 24,169 blocks
    656 ==947393==         suppressed: 0 bytes in 0 blocks
    657 ==947393== Rerun with --leak-check=full to see details of leaked memory
    658 ==947393==
    659 ==947393== For lists of detected and suppressed errors, rerun with: -s
    660 ==947393== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
    661 
    662 
    663 real    0m53.077s
    664 user    0m20.121s
    665 sys     0m0.750s
    666 
    667 andrew@deepthought:~$ time valgrind --leak-check=full --tool=memcheck abg -s 0
    668 
    669 ==948338==
    670 ==948338== LEAK SUMMARY:
    671 ==948338==    definitely lost: 0 bytes in 0 blocks
    672 ==948338==    indirectly lost: 0 bytes in 0 blocks
    673 ==948338==      possibly lost: 6,948,380 bytes in 3,271 blocks
    674 ==948338==    still reachable: 4,899,399 bytes in 24,179 blocks
    675 ==948338==         suppressed: 0 bytes in 0 blocks
    676 ==948338== Reachable blocks (those to which a pointer was found) are not shown.
    677 ==948338== To see them, rerun with: --leak-check=full --show-leak-kinds=all
    678 ==948338==
    679 ==948338== For lists of detected and suppressed errors, rerun with: -s
    680 ==948338== ERROR SUMMARY: 2838 errors from 2838 contexts (suppressed: 0 from 0)
    681 
    682 
    683 real    21m9.471s
    684 user    19m30.601s
    685 sys     1m12.207s
    686 
    687 seems like this was just me not closing the window. fixed that.
    688 
    689 ---
    690 
    691 profiling:
    692 
    693 start: 121mb (time 5:15pm after running for ~a minute)
    694 check-in: 121mb (time 8:55pm) -- this is just the default `abg` cmd so not a ton of churn
    695 stopped: 121mb (9:11pm)
    696 
    697 running again w/ s=0:
    698 
    699 ---
    700 
    701 (recompile)
    702 
    703 start: 123mb (time 9:13pm)
    704 end: 123mb (time: 11:00pm)
    705 
    706 ---
    707 
    708 seems good; calling it here. further refactors can be done that'd improve perf, mostly be improving cache utilization by removing unnecessary bits. This is what I'd say:
    709 
    710 - It's unlikely there's an optimization for my hardware that'd improve performance by 2x
    711 - It's unlikely there's an optimization for my hardware that'd improve performance by 5x across the workloads I've evaluated. There are ways to make prim's algorithm much faster on much larger graphs, but given what I care about, this is likely inconsequential.
    712 - These further optimizations are probably not worth it
    713     - They could make this faster but two things:
    714         1. They would decrease code readability and extensibility a non-trivial amount
    715             - sometimes worth it
    716         2. The improvements wouldn't be meaningful for average usage
    717             - most ppl will be draw constrained. I believe I'm close to what is optimal as it relates to using raylib, without diving straight into opengl
    718 
    719 
    720 ---
    721 
    722 Pygame vs raylib:
    723 
    724 - Not a super great comparison
    725 - SDL (pygame) will basically always be slower than opengl (raylib)
    726 
    727 ---
    728 
    729 okay, so, this is a comparison between pygame, rlib w/ python, rlib from c++:
    730 
    731 these are all only rendering deltas, using a texture with rlib, and just doing incremental writes with pygame (functionally the same amount of computation)
    732 
    733 
    734 each is running one iteration of prim, rendering each step, with 100000 edges and 10000 vertices
    735 
    736 (kept each render in the foreground as that makes it slower, disabled picom)
    737 
    738 sleep 1 for each
    739 
    740 max memory is me running /usr/bin/time -v {command}, taking the max resident set size
    741 
    742 py rlib:
    743 
    744 - (render_rlib)
    745     - 24.347371951736843 seconds avg
    746     - 7,884,780,154.947369 avg cycles
    747 - (render_rlib/mem)
    748     - 160.2857894736842 MiB avg
    749 
    750 py pygame:
    751 
    752 - (render_pygame/v...)
    753     - 52.71620356077778 seconds avg
    754     - 12,353,392,498.666666 avg cycles
    755 - (render_pygame/mem)
    756     - 201.89875 MiB avg
    757 
    758 c++ rlib:
    759 
    760 - (benchmarking/final_render/out.txt)
    761     - 23.859886112999998 seconds avg
    762     - 3,796,753,782.428571 avg cycles
    763 - (benchmarking/final_render_mem/out.txt)
    764     - 137.27142857142857 MiB avg
    765 
    766 summary:
    767 
    768 this is strictly rendering information.
    769 
    770 raylib and pygame, with the same algorithm / write count, albeit randomized so not technically the same graphs, but each with multiple runs, we find:
    771 
    772 - pygame used ~1.25x the memory of raylib
    773 - pygame ran ~2.15x slower than raylib
    774 
    775 python vs c++ (normalized with raylib)
    776 
    777 - python used 1.15x more memory than c++ (c++ not really optimized for memory as it used OO paradigm that increasde overhead relative to python, but still better)
    778 - python version ran ~1.02x slower than c++ version
    779     - the overhead of computation for the graph is trivial relative to the rendering, which was gpu bound in both
    780 - python spent ~3.25x more cpu cycles than the c++ version
    781     - again, gpu bound, so this doesn't change max fps, but does increase energy usage, espeically when considering this is to run in the background for a long time
    782 
    783 without rendering:
    784 
    785 since I didn't do the heap-push optimization in the python code, I'll compare the comparable c++ implementation without that optimization which improved perf by ~2x.
    786 
    787 - benchmarking/final_no_render_v_200000_e_2000000_s_0/
    788     - 2.118 seconds
    789 - no_render_final/v200000e2000000
    790     - 20.253333333333334 seconds
    791 
    792 - c++ ran ~9.56x faster
    793 
    794 takeaway:
    795 
    796 - the overhead for the python raylib bindings aren't that encumbering. The default pygame ones are (pygame -> sdl -> opengl), instead of raylib -> opengl.
    797 - rendering performance is excellerated greatly by re-using texture maps
    798 - raylib -> screenshot is far faster when going -> bmp instead of other image file formats
    799 
    800 ---
    801 
    802 headline: