blog

Personal blog
git clone git://git.laack.co/blog.git
Log | Files | Refs

abg.md (26561B)


      1 # abg
      2 
      3 this is associated with the preliminary post titled: "to make a cool background"
      4 
      5 # To Make a Cool Background
      6 
      7 I wrote a visualization program for prim's algorithm that induces a minimum spanning tree (MST) on a procedurally generated graph. How does one go about making this their background and screensaver with X11 / dwm?
      8 
      9 ## MST
     10 
     11 IMPLEMENTATION DETAILS
     12 
     13 ## Background making
     14 
     15 [https://github.com/python-xlib/python-xlib](https://github.com/python-xlib/python-xlib.md)
     16 
     17 ```python3
     18 os.environ["SDL_VIDEODRIVER"] = "dummy"
     19 ```
     20 
     21 why not just use screenshots in a shared location? 
     22 
     23 indeed, why not?
     24 
     25 ## fuck you and your bullshit.
     26 
     27 Benchmarking
     28 
     29 ## c -> c++
     30 
     31 realized I needed a heap.
     32 
     33 ## bmp vs png vs jpg
     34 
     35 bmp is crazy fucking fast (benchmark)
     36 
     37 (looked in the code for supported file formats, looked in feh -> lib(whatever) for support)
     38 
     39 note larger file size.
     40 
     41 this is interesting, look further.
     42 
     43 ### patched upstream xl
     44 
     45 ### x11
     46 
     47 x11 is weird with this because they are kind hacky in that they just put up a window and take away your inputs, forcing them to go to that window.
     48 
     49 (should learn more about this)
     50 
     51 ### Benchmarking
     52 
     53 uhhh... is it actually that fast?
     54 
     55 wait; did this just optimize all of my code?
     56 
     57 Well yes, but actually no. 
     58 
     59 adding:
     60 
     61 std::cout << g.toString() << std::endl;
     62 
     63 gives ~ the same perf (minus the amount of time it takes to send so much data to stdout)
     64 
     65 (this would require the full traversal because toString is dependent upon prior prim algo steps being performed otherwise the traversal status for edges and vertices would be incorrect).
     66 
     67 (fixed a bug in the python code slowing it down)
     68 
     69 see no_render_2_fixed_itr.
     70 
     71 ## C++ benchmarking
     72 
     73 330 ms for init stuff w/ graph
     74 1405 ms for iteration
     75 
     76 broken further down:
     77 
     78 average iteration time is ~0ms, reaching ~20ms for later stages. Over time the percentage of valid edges declines.
     79 
     80 added getUnvisitedEdges function to return only non-traversed edges
     81 
     82 Init time: 332
     83 Loop time: 1344
     84 Init time: 327
     85 Loop time: 1350
     86 Init time: 322
     87 Loop time: 1374
     88 
     89 still has a similar issue as before; degredation in perf towards the end bc things are pushed earlier on and then get invalidated later on.
     90 
     91 that's basically all I could squeeze out without changes to the algo to be eager instead of lazy wrt tracking vertices. Basically, what's the shortest distance to a given vertex instead of what's the edge that we have that's shortest. 
     92 
     93 such a refactor though is likely irrelevant right now due to the added cost of 
     94 
     95 ope; realized my function for get edges with unvisited vertices above was wrong. I was just not pushing visited edges, but we really care about vertices that haven't been visited so refactored the edge generation and stuff to get a stable ref to v2index and then push only if that hasn't been visited.
     96 
     97 Init time: 389
     98 Loop time: 706
     99 Init time: 315
    100 Loop time: 671
    101 Init time: 337
    102 Loop time: 692
    103 
    104 now we're cooking with gas.
    105 
    106 tests now fail. bc snapshots. updated and everything seems right again.
    107 
    108 actually the stable things isn't really necessary as this approach that's unstable is functionally the same, but safer in case there are changes to the IR of edges.
    109 
    110 Init time: 334
    111 Loop time: 694
    112 Init time: 321
    113 Loop time: 668
    114 Init time: 323
    115 Loop time: 701
    116 
    117 now we find this:
    118 
    119 (see benchmarking/no_render_v_100000_e_1000000_s_0/out1.csv)
    120 
    121 same optimization performed for python code so that's then down to 8.6 seconds ~~~ see andrew@deepthought:no_render_3_fixed_itr_fixed_check$
    122 
    123 at this point it's worth worrying about the return for the get untraversed edges for vertices not traversed. after that:
    124 
    125 Init time: 314
    126 Loop time: 678
    127 Init time: 312
    128 Loop time: 671
    129 Init time: 311
    130 Loop time: 680
    131 
    132 and with the optimization for edge ordering:
    133 
    134 Init time: 311
    135 Loop time: 659
    136 Init time: 333
    137 Loop time: 673
    138 Init time: 320
    139 Loop time: 709
    140 Init time: 311
    141 Loop time: 674
    142 
    143 this is still too fucking slow.
    144 
    145 what's making the graph induction so incredibly slow?
    146 
    147 just graph creation:
    148 
    149 graph create: 317
    150 Loop time: 691
    151 graph create: 314
    152 Loop time: 657
    153 graph create: 311
    154 Loop time: 665
    155 
    156 reserving instead of pushing:
    157 
    158 graph create: 314
    159 Loop time: 636
    160 graph create: 324
    161 Loop time: 688
    162 graph create: 315
    163 Loop time: 693
    164 
    165 replaced map with vector where we were using indices to index into the map (dumb):
    166 
    167 graph create: 174
    168 Loop time: 654
    169 graph create: 179
    170 Loop time: 610
    171 graph create: 184
    172 Loop time: 618
    173 graph create: 185
    174 Loop time: 622
    175 
    176 ok, let's now run the full benchmarks again:
    177 
    178 benchmarking/no_render_only_grab_unvisited_edges_vector_instead_of_map_v_100000_e_1000000_s_0/out1.csv
    179 
    180 0.04,0.79,0.84
    181 0.03,0.79,0.84
    182 0.04,0.77,0.82
    183 
    184 this was kind of just stalling though becaus I'm out of my depth as it relates to rendering, the larger perf issue.
    185 
    186 back to that. 
    187 
    188 here are where this stands w/ 1000 v and 10_000 edges
    189 
    190 prior was ~34s per iteration (final col)
    191 
    192 this is basically the same:
    193 
    194 benchmarking/better_render_v_1000_e_10000_s_0/out1.csv
    195 
    196 (snippet from above file)
    197 
    198 0.26,10.71,34.22
    199 0.29,10.64,33.24
    200 0.31,10.62,33.57
    201 0.28,10.53,33.56
    202 
    203 fine. we'll fix the raylib shit. (queue montage)
    204 
    205 make debug-build && time ./abg.out -s 0 --edges 2000 --vertices 200
    206 
    207 real    0m1.948s
    208 user    0m0.529s
    209 sys     0m0.060s
    210 
    211 ~20 milliseconds per render call w/ make debug-build && time ./abg.out -s 0 --edges 20000 --vertices 2000
    212 
    213 Total single iteration cost: 65
    214 Amount of time calling render: 18
    215 Total single iteration cost: 65
    216 Amount of time calling render: 18
    217 Total single iteration cost: 65
    218 Amount of time calling render: 18
    219 Total single iteration cost: 65
    220 Amount of time calling render: 18
    221 Total single iteration cost: 65
    222 Amount of time calling render: 19
    223 Total single iteration cost: 65
    224 Amount of time calling render: 18
    225 Total single iteration cost: 65
    226 Amount of time calling render: 18
    227 Total single iteration cost: 65
    228 
    229 huh; what's the other iteration cost?
    230 
    231 
    232     while (!WindowShouldClose() && toVisit.size() != 0) {
    233         auto startLoop = std::chrono::steady_clock::now();
    234         BeginDrawing();
    235         ClearBackground(BLACK);
    236         auto startRender = std::chrono::steady_clock::now();
    237         g.render();
    238         auto endRender = std::chrono::steady_clock::now();
    239         auto diff = std::chrono::duration_cast<std::chrono::milliseconds>(endRender - startRender);
    240         std::cout << "Amount of time calling render: " << diff.count() << std::endl;
    241         EndDrawing();
    242 
    243         // since we wait sleepTime here, the bg render has a render delta of
    244         // at minimum sleepTime when switching tags in dwm, this is rather
    245         // annoying because the screen doesn't repaint until the sleep time
    246         // passes, which results in artifacts on screen.
    247 
    248         // despite this, calling render a lot of times is rather intensive
    249         // (at least on my hardware) and so this tradeoff is accepted for
    250         // now, unless there's a simple approach that allows for preemption
    251 
    252         //usleep((int)(sleepTime * 1000000));
    253 
    254         oneStepPrim(toVisit, visitedIndices, g);
    255         auto endLoop = std::chrono::steady_clock::now();
    256         auto loopDiff = std::chrono::duration_cast<std::chrono::milliseconds>(endLoop - startLoop);
    257         std::cout << "Total single iteration cost: " << loopDiff.count() << std::endl;
    258 
    259     }
    260 
    261 so it's somewhere in that first block; in the render area. the actual computation cost at the end is inconsequential. 
    262 
    263 begindrawing is ~free.
    264 
    265 clear background is ~free.
    266 
    267 ~46 of 65ms are spent on enddrawing. 
    268 
    269 so two things:
    270 
    271 19ms on render
    272 46ms on enddrawing
    273 
    274 what about when the numbers are much bigger (100_000 edges, 10_000 vertices)?
    275 
    276 Render time: 56
    277 End Drawing time: 237
    278 Total single iteration cost: 316
    279 
    280 the other 23 ms is the prim step + begin drawing / clear background.
    281 
    282 can we just not clear the screen? does that help? nope. doesn't do anything.
    283 
    284 target 60 fps?
    285 
    286 nope.
    287 
    288 ahh; the slowness is being realized later on. while render is cool with me double calling on edges; this is felt when enddrawing is called as the gpu does the rendering and stuff at that point. 
    289 
    290 what happens if I use the better single time render method?
    291 
    292 before:
    293 
    294 Render time: 62
    295 End Drawing time: 244
    296 Total single iteration cost: 306
    297 Render time: 60
    298 End Drawing time: 245
    299 Total single iteration cost: 305
    300 Render time: 60
    301 End Drawing time: 245
    302 Total single iteration cost: 305
    303 Render time: 55
    304 End Drawing time: 252
    305 Total single iteration cost: 307
    306 Render time: 60
    307 End Drawing time: 245
    308 Total single iteration cost: 305
    309 Render time: 58
    310 End Drawing time: 247
    311 Total single iteration cost: 306
    312 
    313 after:
    314 
    315 Total single iteration cost: 158
    316 Render time: 46
    317 End Drawing time: 112
    318 Total single iteration cost: 159
    319 Render time: 46
    320 End Drawing time: 112
    321 Total single iteration cost: 159
    322 Render time: 45
    323 End Drawing time: 113
    324 Total single iteration cost: 159
    325 Render time: 46
    326 End Drawing time: 111
    327 Total single iteration cost: 158
    328 Render time: 46
    329 End Drawing time: 112
    330 Total single iteration cost: 158
    331 Render time: 46
    332 End Drawing time: 113
    333 
    334 hell yeah.
    335 
    336 why so slow still?
    337 
    338 Like this is only ~100_000 and ~10_000 vertices so like why so slow?
    339 
    340 benchmarked now:
    341 
    342 benchmarking/simpler_render_v_1000_e_10000_s_0/out1.csv
    343 0.29,7.77,18.96
    344 0.27,7.80,19.17
    345 0.29,7.90,19.17
    346 0.31,7.76,19.05
    347 0.29,7.72,18.68
    348 0.30,7.90,18.56
    349 0.27,7.78,18.80
    350 0.27,7.77,18.74
    351 0.29,7.69,18.77
    352 0.32,7.67,18.85
    353 
    354 alright then. unfortunately, we're only calling the draw function once per vertex and edge now. we could change the rendering to only render connections, this would increase speed a lot, but it also makes the animation look less cool so fuck that. 
    355 
    356 could we draw in the background? like create a new thread that doesn't block us and then wait until that's done once we get back?
    357 
    358 well maybe, but the issue is I don't know how well this handles drawing while there's a queue of messages still being sent. 
    359 
    360 std::thread t(endDrawing);
    361 t.join();
    362 
    363 vs endDrawing()
    364 
    365 result in entirely different outcomes.
    366 
    367 the second sort of renders some stuff, but gets totally fucked up and then just black screens.
    368 
    369 due to an import? no this shit's just fucked.
    370 
    371 opengl affinity.
    372 
    373 can prebake the graph though and then only render additions.
    374 
    375 after:
    376 
    377 0.19,1.77,3.86
    378 0.17,1.76,3.81
    379 0.19,1.76,3.82
    380 0.18,1.75,3.85
    381 0.18,1.71,3.86
    382 0.19,1.68,3.83
    383 0.18,1.72,3.83
    384 
    385 holy shit. can we make this faster?
    386 
    387 of fucking course we can. track if a traversed state has been rendered and skip it if it has, using this to update our incremental texture. we then render that. 
    388 
    389 benchmarking/cache_first_use_blank_graph_v_1000_e_10000_s_0/out1.csv
    390 
    391 0.17,0.65,3.06
    392 0.18,0.63,3.05
    393 0.18,0.66,3.05
    394 0.18,0.64,3.05
    395 0.18,0.64,3.04
    396 0.17,0.63,3.02
    397 0.18,0.62,3.04
    398 0.19,0.64,3.05
    399 0.17,0.65,3.04
    400 0.17,0.66,3.03
    401 0.18,0.62,3.04
    402 0.17,0.63,3.03
    403 
    404 this is too boring; let's turn this shit up.
    405 
    406 how fast are we relative to the python bloatware webshit?
    407 
    408 benchmarking/cache_first_use_blank_graph_v_10000_e_100000_s_0/out1.csv
    409 
    410 1.40,28.92,35.76
    411 1.53,29.30,35.95
    412 
    413 (notice the edges and vertices)
    414 
    415 vs python:
    416 
    417 1.33,10730.71,10772.74
    418 
    419 that's only >300x slower for the python code so the average python developer would totally be cool with shipping it. Job done, webshit: built. 
    420 
    421 alright then. enough messing around. We really should have a queue for items that are to be rendered so we don't have to iterate over all vertices and edges, checking if they have the 'rendered' flag set. 
    422 
    423 after doing this we find:
    424 
    425 benchmarking/track_to_render_v_10000_e_100000_s_0/out1.csv
    426 
    427 1.50,3.12,29.43
    428 1.51,3.20,29.67
    429 1.57,3.08,30.00
    430 1.57,3.04,30.07
    431 
    432 35.855000000000004 / 29.7925 = 1.2034908114458338
    433 
    434 okay, so 20 percent faster. still too fucking slow.
    435 
    436 i'm also getting sick of this bs chrono / time shit. time to bring in perf.
    437 
    438  3.20%           395  abg.out  libraylib.so.6.0.0            [.] rlVertex3f
    439  2.08%           338  abg.out  abg.out                       [.] Edge::operator>(Edge const&) const
    440  1.55%           245  abg.out  abg.out                       [.] oneStepPrim(std::priority_queue<Edge, std::vector<Edge, st>
    441  1.41%           255  abg.out  libraylib.so.6.0.0            [.] rlDrawRenderBatch
    442  1.22%           225  abg.out  libc.so.6                     [.] 0x0000000000185dde
    443  0.85%           129  abg.out  libraylib.so.6.0.0            [.] DrawCircleSector
    444  0.79%           136  abg.out  libc.so.6                     [.] pthread_mutex_lock
    445  0.74%           133  abg.out  libc.so.6                     [.] ioctl
    446  0.64%           115  abg.out  libgallium-26.2.2-arch1.1.so  [.] 0x00000000007f8138
    447  0.62%           113  abg.out  libraylib.so.6.0.0            [.] PollInputEvents
    448  0.57%            91  abg.out  libc.so.6                     [.] cfree
    449  0.56%            95  abg.out  libc.so.6                     [.] malloc
    450  0.54%            27  abg.out  libc.so.6                     [.] 0x00000000001866c9
    451  0.53%            99  abg.out  libgallium-26.2.2-arch1.1.so  [.] 0x000000000180c45c
    452 
    453 alright so raylib is spending a decent amount of time on Vertex3f though not a crazy amount. Comparisons between edges are common; not surprising as we are using a heap with lots of elements towards the end, as we've seen. prim's algorithm generally taking some time, makes sense.
    454 
    455 render drawing is rather low, but within reason. This is rather disappointing. It's nice when it's like, no, dude, you spent 99.95% of your time in one function. 
    456 
    457 looking at the call stack:
    458 
    459 |--14.80%--EndTextureMode
    460 |--58.02%--EndDrawing
    461 --1.62%--Graph::render()
    462 
    463 well. huh. end texture mode is where I add things though the function itself is just clearing the queue. 
    464 
    465 It's a bit surprising enddrawing is more expensive since it just loads in the texture I've baked. 
    466 
    467 ---
    468 
    469 I shall call this ~done. final findings with 
    470 
    471 rendered below is using     vertices = 10_000, edges = 100_000
    472 
    473 unrendered below is using   vertices = 200_000, edges = 2_000_000 
    474 
    475 - python (with overdraw mostly fixing + texture map)
    476     - rendered 
    477         - 58.758 seconds
    478     - unrendered
    479         - 20.253333333333334 seconds
    480 
    481 - c++ (with overdraw fixing + texture map)
    482     - rendered
    483         - - 29.7925 seconds
    484 
    485     - unrendered
    486         - 2.118 seconds
    487 
    488 
    489 benchmarking/final_no_render_v_200000_e_2000000_s_0/
    490 
    491 takeaway:
    492 
    493 the rendered python implementation w/ 10_000 v and 100_000 edges using perf:
    494 
    495 Performance counter stats for 'python3 prim.py':
    496 
    497                  0      context-switches:u               #      0.0 cs/sec  cs_per_second
    498                  0      cpu-migrations:u                 #      0.0 migrations/sec  migrations_per_second
    499             14,233      page-faults:u                    #    252.0 faults/sec  page_faults_per_second
    500          56,487.57 msec task-clock:u                     #      nan CPUs  CPUs_utilized
    501         30,763,483      branch-misses:u                  #      0.7 %  branch_miss_rate         (88.90%)
    502      4,305,442,939      branches:u                       #     76.2 M/sec  branch_frequency     (88.90%)
    503    128,137,043,750      cpu-cycles:u                     #      2.3 GHz  cycles_frequency       (88.90%)
    504     18,736,054,409      instructions:u                   #      0.1 instructions  insn_per_cycle  (88.85%)
    505             TopdownL1 #      0.1 %  tma_backend_bound
    506                                                          #     98.9 %  tma_bad_speculation      (88.87%)
    507                                                          #      0.5 %  tma_frontend_bound       (77.81%)
    508                                                          #      0.5 %  tma_retiring             (88.92%)
    509 
    510       59.936596362 seconds time elapsed
    511 
    512       52.628026000 seconds user
    513        1.492521000 seconds sys
    514 
    515 gpu constrained though. 
    516 
    517 the rendered c++ implementation w/ 10_000 v and 100_000 edges using perf:
    518 
    519  Performance counter stats for './abg.out -s 0 --vertices 10000 --edges 100000':
    520 
    521                  0      context-switches:u               #      0.0 cs/sec  cs_per_second
    522                  0      cpu-migrations:u                 #      0.0 migrations/sec  migrations_per_second
    523              9,363      page-faults:u                    #   1819.4 faults/sec  page_faults_per_second
    524           5,146.21 msec task-clock:u                     #      nan CPUs  CPUs_utilized
    525         28,144,835      branch-misses:u                  #      5.9 %  branch_miss_rate         (89.55%)
    526        473,319,266      branches:u                       #     92.0 M/sec  branch_frequency     (88.64%)
    527      3,973,233,282      cpu-cycles:u                     #      0.8 GHz  cycles_frequency       (88.55%)
    528      2,925,854,560      instructions:u                   #      0.7 instructions  insn_per_cycle  (89.12%)
    529             TopdownL1 #      2.7 %  tma_backend_bound
    530                                                          #     93.6 %  tma_bad_speculation      (88.66%)
    531                                                          #      0.9 %  tma_frontend_bound       (77.75%)
    532                                                          #      2.8 %  tma_retiring             (88.67%)
    533 
    534       29.577542431 seconds time elapsed
    535 
    536        2.925597000 seconds user
    537        2.094917000 seconds sys
    538 
    539 32x less instructions than the python version. Neither is cpu constrained, but this is a non-trivial amount of overhead from the python version; c++ version also had 7x higher instructions per cycle. 
    540 
    541 gpu constrained too.
    542 
    543 memory overhead?
    544 
    545 c++ rendered sits at ~171mb-175mb. it's not great, hasn't been optimized, def room for improvement. 
    546 
    547 python rendered sits at ~240mb-248mb
    548 
    549 the default invocation method for the c++ implementation I use as my lock screen is running at 160mb. This is rather annoying, but also, it's doing a bunch of rendering stuff so IG that's fine, and this does improve the computational cost in terms of cycles so...
    550 
    551 huh... after 2 hours it's chilling at 163mb (TODO).
    552 
    553 bc it's always holding onto at least one frame which is 5120x1440 pixels which is ~21mb minimum (assuming r g and b bytes per pixel.)
    554 
    555 an eager approach for this would be better, but c++ stl doesn't have an indexed priority queue, so I'll just retcon what I have. 
    556 
    557 basically, I just want to minimize the useless things I push to the queue. One way to do this is to track the minimum weighted edge with an untraversed vertex and then updating this and only pushing edges with it when they are < that weight. 
    558 
    559 before:
    560 
    561 benchmarking/final_no_render_v_200000_e_2000000_s_0/
    562 
    563 0.08,2.03,2.12
    564 0.09,2.02,2.12
    565 0.09,2.00,2.11
    566 0.09,2.00,2.11
    567 0.08,1.99,2.09
    568 0.07,1.99,2.08
    569 0.10,2.00,2.11
    570 0.09,2.00,2.11
    571 0.09,2.07,2.18
    572 0.06,2.08,2.15
    573 
    574 
    575 after:
    576 
    577 benchmarking/final_min_no_render_v_200000_e_2000000_s_0/out.csv
    578 
    579 0.06,0.97,1.04
    580 0.06,0.93,1.00
    581 0.07,0.95,1.03
    582 0.09,0.92,1.02
    583 0.08,0.94,1.02
    584 0.08,0.94,1.03
    585 0.08,0.94,1.03
    586 0.06,0.95,1.02
    587 0.08,0.93,1.02
    588 0.07,0.95,1.02
    589 0.07,0.95,1.03
    590 0.07,0.95,1.03
    591 0.08,0.95,1.04
    592 0.09,0.93,1.03
    593 0.07,0.98,1.06
    594 0.07,0.95,1.04
    595 0.08,0.94,1.03
    596 0.07,0.96,1.04
    597 0.07,0.95,1.04
    598 0.07,1.00,1.08
    599 
    600 this doesn't really impact time wrt rendered because that is basically all spent on rendering not computation. 
    601 
    602 do I have a memory leak?
    603 
    604 
    605 
    606 > time valgrind --tool=memcheck abg
    607 
    608 ==944657== Process terminating with default action of signal 2 (SIGINT)
    609 ==944657==    at 0x50D3952: __syscall_cancel_arch (syscall_cancel.S:56)
    610 ==944657==    by 0x5117AEC: internal_syscall_cancel (sysdep-cancel.h:53)
    611 ==944657==    by 0x5117AEC: clock_nanosleep@@GLIBC_2.17 (clock_nanosleep.c:48)
    612 ==944657==    by 0x5123F26: nanosleep (nanosleep.c:25)
    613 ==944657==    by 0x51532E9: usleep (usleep.c:31)
    614 ==944657==    by 0x40065B8: main (in /usr/local/bin/abg)
    615 ==944657==
    616 ==944657== HEAP SUMMARY:
    617 ==944657==     in use at exit: 11,776,171 bytes in 27,368 blocks
    618 ==944657==   total heap usage: 86,922 allocs, 59,554 frees, 29,848,973 bytes allocated
    619 ==944657==
    620 ==944657== LEAK SUMMARY:
    621 ==944657==    definitely lost: 0 bytes in 0 blocks
    622 ==944657==    indirectly lost: 0 bytes in 0 blocks
    623 ==944657==      possibly lost: 6,876,540 bytes in 3,237 blocks
    624 ==944657==    still reachable: 4,899,631 bytes in 24,131 blocks
    625 ==944657==         suppressed: 0 bytes in 0 blocks
    626 ==944657== Rerun with --leak-check=full to see details of leaked memory
    627 ==944657==
    628 ==944657== For lists of detected and suppressed errors, rerun with: -s
    629 ==944657== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
    630 
    631 
    632 real    12m18.560s
    633 user    0m23.848s
    634 sys     0m0.969s
    635 
    636 andrew@deepthought:~$ time valgrind --tool=memcheck abg
    637 ==947393== Memcheck, a memory error detector
    638 ==947393== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al.
    639 ==947393== Using Valgrind-3.25.1 and LibVEX; rerun with -h for copyright info
    640 ==947393== Command: abg
    641 ==947393==
    642 ^C==947393==
    643 ==947393== Process terminating with default action of signal 2 (SIGINT)
    644 ==947393==    at 0x50D3952: __syscall_cancel_arch (syscall_cancel.S:56)
    645 ==947393==    by 0x5117AEC: internal_syscall_cancel (sysdep-cancel.h:53)
    646 ==947393==    by 0x5117AEC: clock_nanosleep@@GLIBC_2.17 (clock_nanosleep.c:48)
    647 ==947393==    by 0x5123F26: nanosleep (nanosleep.c:25)
    648 ==947393==    by 0x51532E9: usleep (usleep.c:31)
    649 ==947393==    by 0x40065B8: main (in /usr/local/bin/abg)
    650 ==947393==
    651 ==947393== HEAP SUMMARY:
    652 ==947393==     in use at exit: 11,656,499 bytes in 27,404 blocks
    653 ==947393==   total heap usage: 64,820 allocs, 37,416 frees, 24,650,054 bytes allocated
    654 ==947393==
    655 ==947393== LEAK SUMMARY:
    656 ==947393==    definitely lost: 0 bytes in 0 blocks
    657 ==947393==    indirectly lost: 0 bytes in 0 blocks
    658 ==947393==      possibly lost: 6,744,916 bytes in 3,235 blocks
    659 ==947393==    still reachable: 4,911,583 bytes in 24,169 blocks
    660 ==947393==         suppressed: 0 bytes in 0 blocks
    661 ==947393== Rerun with --leak-check=full to see details of leaked memory
    662 ==947393==
    663 ==947393== For lists of detected and suppressed errors, rerun with: -s
    664 ==947393== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
    665 
    666 
    667 real    0m53.077s
    668 user    0m20.121s
    669 sys     0m0.750s
    670 
    671 andrew@deepthought:~$ time valgrind --leak-check=full --tool=memcheck abg -s 0
    672 
    673 ==948338==
    674 ==948338== LEAK SUMMARY:
    675 ==948338==    definitely lost: 0 bytes in 0 blocks
    676 ==948338==    indirectly lost: 0 bytes in 0 blocks
    677 ==948338==      possibly lost: 6,948,380 bytes in 3,271 blocks
    678 ==948338==    still reachable: 4,899,399 bytes in 24,179 blocks
    679 ==948338==         suppressed: 0 bytes in 0 blocks
    680 ==948338== Reachable blocks (those to which a pointer was found) are not shown.
    681 ==948338== To see them, rerun with: --leak-check=full --show-leak-kinds=all
    682 ==948338==
    683 ==948338== For lists of detected and suppressed errors, rerun with: -s
    684 ==948338== ERROR SUMMARY: 2838 errors from 2838 contexts (suppressed: 0 from 0)
    685 
    686 
    687 real    21m9.471s
    688 user    19m30.601s
    689 sys     1m12.207s
    690 
    691 seems like this was just me not closing the window. fixed that.
    692 
    693 ---
    694 
    695 profiling:
    696 
    697 start: 121mb (time 5:15pm after running for ~a minute)
    698 check-in: 121mb (time 8:55pm) -- this is just the default `abg` cmd so not a ton of churn
    699 stopped: 121mb (9:11pm)
    700 
    701 running again w/ s=0:
    702 
    703 ---
    704 
    705 (recompile)
    706 
    707 start: 123mb (time 9:13pm)
    708 end: 123mb (time: 11:00pm)
    709 
    710 ---
    711 
    712 seems good; calling it here. further refactors can be done that'd improve perf, mostly be improving cache utilization by removing unnecessary bits. This is what I'd say:
    713 
    714 - It's unlikely there's an optimization for my hardware that'd improve performance by 2x
    715 - It's unlikely there's an optimization for my hardware that'd improve performance by 5x across the workloads I've evaluated. There are ways to make prim's algorithm much faster on much larger graphs, but given what I care about, this is likely inconsequential.
    716 - These further optimizations are probably not worth it
    717     - They could make this faster but two things:
    718         1. They would decrease code readability and extensibility a non-trivial amount
    719             - sometimes worth it
    720         2. The improvements wouldn't be meaningful for average usage
    721             - most ppl will be draw constrained. I believe I'm close to what is optimal as it relates to using raylib, without diving straight into opengl
    722 
    723 
    724 ---
    725 
    726 Pygame vs raylib:
    727 
    728 - Not a super great comparison
    729 - SDL (pygame) will basically always be slower than opengl (raylib)
    730 
    731 ---
    732 
    733 okay, so, this is a comparison between pygame, rlib w/ python, rlib from c++:
    734 
    735 these are all only rendering deltas, using a texture with rlib, and just doing incremental writes with pygame (functionally the same amount of computation)
    736 
    737 
    738 each is running one iteration of prim, rendering each step, with 100000 edges and 10000 vertices
    739 
    740 (kept each render in the foreground as that makes it slower, disabled picom)
    741 
    742 sleep 1 for each
    743 
    744 max memory is me running /usr/bin/time -v {command}, taking the max resident set size
    745 
    746 py rlib:
    747 
    748 - (render_rlib)
    749     - 24.347371951736843 seconds avg
    750     - 7,884,780,154.947369 avg cycles
    751 - (render_rlib/mem)
    752     - 160.2857894736842 MiB avg
    753 
    754 py pygame:
    755 
    756 - (render_pygame/v...)
    757     - 52.71620356077778 seconds avg
    758     - 12,353,392,498.666666 avg cycles
    759 - (render_pygame/mem)
    760     - 201.89875 MiB avg
    761 
    762 c++ rlib:
    763 
    764 - (benchmarking/final_render/out.txt)
    765     - 23.859886112999998 seconds avg
    766     - 3,796,753,782.428571 avg cycles
    767 - (benchmarking/final_render_mem/out.txt)
    768     - 137.27142857142857 MiB avg
    769 
    770 summary:
    771 
    772 this is strictly rendering information.
    773 
    774 raylib and pygame, with the same algorithm / write count, albeit randomized so not technically the same graphs, but each with multiple runs, we find:
    775 
    776 - pygame used ~1.25x the memory of raylib
    777 - pygame ran ~2.15x slower than raylib
    778 
    779 python vs c++ (normalized with raylib)
    780 
    781 - python used 1.15x more memory than c++ (c++ not really optimized for memory as it used OO paradigm that increasde overhead relative to python, but still better)
    782 - python version ran ~1.02x slower than c++ version
    783     - the overhead of computation for the graph is trivial relative to the rendering, which was gpu bound in both
    784 - python spent ~3.25x more cpu cycles than the c++ version
    785     - again, gpu bound, so this doesn't change max fps, but does increase energy usage, espeically when considering this is to run in the background for a long time
    786 
    787 without rendering:
    788 
    789 since I didn't do the heap-push optimization in the python code, I'll compare the comparable c++ implementation without that optimization which improved perf by ~2x.
    790 
    791 - benchmarking/final_no_render_v_200000_e_2000000_s_0/
    792     - 2.118 seconds
    793 - no_render_final/v200000e2000000
    794     - 20.253333333333334 seconds
    795 
    796 - c++ ran ~9.56x faster
    797 
    798 takeaway:
    799 
    800 - the overhead for the python raylib bindings aren't that encumbering. The default pygame ones are (pygame -> sdl -> opengl), instead of raylib -> opengl.
    801 - rendering performance is excellerated greatly by re-using texture maps
    802 - raylib -> screenshot is far faster when going -> bmp instead of other image file formats
    803 
    804 ---
    805 
    806 headline:
    807