http://simonwillison.net/2024-07-26T13:06:38+00:00Simon Willison2024-07-26T13:06:38+00:002024-07-26T13:06:38+00:00https://simonwillison.net/2024/Jul/26/did-you-know-about-instruments/#atom-tag

<p><strong><a href="https://registerspill.thorstenball.com/p/did-you-know-about-instruments">Did you know about Instruments?</a></strong></p> Thorsten Ball shows how the macOS Instruments app (installed as part of Xcode) can be used to run a CPU profiler against <em>any</em> application - not just code written in Swift/Objective C.</p> <p>I tried this against a Python process running <a href="https://llm.datasette.io/">LLM</a> executing a Llama 3.1 prompt with my new <a href="https://github.com/simonw/llm-gguf">llm-gguf</a> plugin and captured this:</p> <p><img alt="Screenshot of a deep nested stack trace showing _PyFunction_Vectorcall from python3.10 calling PyCFuncPtr_call _ctypes.cpython-310-darwin.so which then calls ggml_ methods in libggml.dylib" src="https://static.simonwillison.net/static/2024/instruments-ggml.jpg" /> <p><small></small>Via <a href="https://lobste.rs/s/kr9od0/did_you_know_about_instruments">lobste.rs</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/observability">observability</a></p>

2022-06-13T19:40:43+00:002022-06-13T19:40:43+00:00https://simonwillison.net/2022/Jun/13/making-code-faster/#atom-tag

<p><strong><a href="https://www.tbray.org/ongoing/When/202x/2022/06/10/Quamina-Optimizing">Making Code Faster</a></strong></p> Tim Bray’s detailed guide to using the Go profiler. <p>Tags: <a href="https://simonwillison.net/tags/go">go</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/tim-bray">tim-bray</a></p>

2021-03-01T19:12:03+00:002021-03-01T19:12:03+00:00https://simonwillison.net/2021/Mar/1/gta-loading-times/#atom-tag

<p><strong><a href="https://nee.lv/2021/02/28/How-I-cut-GTA-Online-loading-times-by-70/">How I cut GTA Online loading times by 70%</a></strong></p> Incredible debugging war story: t0st was fed up of waiting six minutes (!) for GTA Online to load on their PC, so they used a host of devious debugging tricks to try and figure out what was going on. It turned out the game was loading a 10MB JSON file detailing all of the available in-game purchases, but inefficient JSON parsing meant it was pegging an entire CPU for 4 minutes mainly running the strlen() C function. Despite not having access to source code or debugging symbols t0st figured out the problem and managed to inject a custom DLL that hooked some internal functions and dropped load times down from 6m down to to 1m50s! <p><small></small>Via <a href="https://news.ycombinator.com/item?id=26296339">Hacker News</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/debugger">debugger</a>, <a href="https://simonwillison.net/tags/hacking">hacking</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a></p>

2019-07-24T08:25:07+00:002019-07-24T08:25:07+00:00https://simonwillison.net/2019/Jul/24/memory-profiler/#atom-tag

<p><strong><a href="https://github.com/dogsheep/healthkit-to-sqlite/issues/7">Using memory-profiler to debug excessive memory usage in healthkit-to-sqlite</a></strong></p> This morning I figured out how to use the memory-profiler module (and mprof command line tool) to debug memory usage of Python processes. I added the details, including screenshots, to this GitHub issue. It helped me knock down RAM usage for my healthkit-to-sqlite from 2.5GB to just 80MB by making smarter usage of the ElementTree pull parser. <p>Tags: <a href="https://simonwillison.net/tags/elementtree">elementtree</a>, <a href="https://simonwillison.net/tags/memory">memory</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/xml">xml</a></p>

2018-03-21T20:56:50+00:002018-03-21T20:56:50+00:00https://simonwillison.net/2018/Mar/21/flame-graphs/#atom-tag

<p><strong><a href="https://rbspy.github.io/profiling-guide/using-flamegraphs.html">Using flamegraphs</a></strong></p> I really like flamegraphs as a profiling tool—we have support for them baked into our Tikibar debugging toolbar at Eventbrite—but interpreting them isn’t particularly intuitive on first glance. Julia Evans has put together a great explanation of how to read them as part of the documentation for her rbspy Ruby profiler. <p>Tags: <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/julia-evans">julia-evans</a></p>

2017-10-28T01:06:49+00:002017-10-28T01:06:49+00:00https://simonwillison.net/2017/Oct/28/profiling/#atom-tag

<p><strong><a href="https://github.com/what-studio/profiling">profiling</a></strong></p> “An interactive continuous Python profiler”. This is really neat—simply run “profiling myscript.py” to get an interactive, navigable console-based profile inspector at the end of your script... or run “profiling live-profile mywebverver.py” to see a live, updating profile of a long-running process. Has options for statistical profiling as well, which has a much lower overhead in exchange for a less accurate view of what is going on. <p>Tags: <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/python">python</a></p>

2017-10-05T15:44:20+00:002017-10-05T15:44:20+00:00https://simonwillison.net/2017/Oct/5/profiler/#atom-tag

<p><strong><a href="https://www.polibyte.com/2017/08/05/building-statistical-profiler/">Building a statistical profiler in python</a></strong></p> Generating flame graphs of production Python code is surprisingly straight-forward. Brian Pitts built a statistical flame graph profiler into our tikibar debugging tool at Eventbrite and it’s proved extremely useful. <p>Tags: <a href="https://simonwillison.net/tags/django">django</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/python">python</a>, <a href="https://simonwillison.net/tags/tikibar">tikibar</a></p>

2009-08-19T11:42:28+00:002009-08-19T11:42:28+00:00https://simonwillison.net/2009/Aug/19/tcpdump/#atom-tag

<p><strong><a href="http://www.xaprb.com/blog/2009/08/18/how-to-find-un-indexed-queries-in-mysql-without-using-the-log/">How to find un-indexed queries in MySQL, without using the log</a></strong></p> Use tcpdump(!) to sniff the MySQL protocol and dump out queries that had the “no index used” bit set. <p><small></small>Via <a href="http://delicious.com/mattb/maatkit+mysql+query+sql+tcpdump">Matt Biddulph</a></small></p> <p>Tags: <a href="https://simonwillison.net/tags/mysql">mysql</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/tcpdump">tcpdump</a></p>

2009-04-29T23:57:26+00:002009-04-29T23:57:26+00:00https://simonwillison.net/2009/Apr/29/building/#atom-tag

<p><strong><a href="http://www.alertdebugging.com/2009/04/29/building-a-better-javascript-profiler-with-webkit/">Building a Better JavaScript Profiler with WebKit</a></strong></p> Clever hack from Francisco Tolmasky which solves the problem of JavaScript profilers showing ? as the name of any anonymous functions. He patched the WebKit profiler to look for a displayName attribute on a function and show that as the function name instead. <p>Tags: <a href="https://simonwillison.net/tags/francisco-tolmasky">francisco-tolmasky</a>, <a href="https://simonwillison.net/tags/javascript">javascript</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/webkit">webkit</a></p>

2009-04-02T10:34:15+00:002009-04-02T10:34:15+00:00https://simonwillison.net/2009/Apr/2/jabbering/#atom-tag

<p><strong><a href="http://happygiraffe.net/blog/2009/04/02/heap-dump-analysis/">Heap Dump Analysis</a></strong></p> Using jmap to dump the JVM’s memory to disk, then analysing it using the visualvm GUI tool. <p>Tags: <a href="https://simonwillison.net/tags/dominic-mitchell">dominic-mitchell</a>, <a href="https://simonwillison.net/tags/heapdump">heapdump</a>, <a href="https://simonwillison.net/tags/java">java</a>, <a href="https://simonwillison.net/tags/jvm">jvm</a>, <a href="https://simonwillison.net/tags/memory">memory</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/visualvm">visualvm</a></p>

2008-10-01T01:20:24+00:002008-10-01T01:20:24+00:00https://simonwillison.net/2008/Oct/1/mysql/#atom-tag

<p><strong><a href="http://dev.mysql.com/tech-resources/articles/using-new-query-profiler.html">Using the New MySQL Query Profiler</a></strong></p> Extremely powerful new feature in MySQL 5.0.37. Definitely something for the Django debug toolbar. <p>Tags: <a href="https://simonwillison.net/tags/django">django</a>, <a href="https://simonwillison.net/tags/django-debug-toolbar">django-debug-toolbar</a>, <a href="https://simonwillison.net/tags/mysql">mysql</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a></p>

2008-06-16T10:20:36+00:002008-06-16T10:20:36+00:00https://simonwillison.net/2008/Jun/16/john/#atom-tag

<p><strong><a href="http://ejohn.org/blog/deep-profiling-jquery-apps/">Deep Profiling jQuery Apps</a></strong></p> Neat plugin from John Resig that monkey-patches most (all?) of the jQuery methods to build up a detailed profile of which methods are being used by a given page. <p>Tags: <a href="https://simonwillison.net/tags/javascript">javascript</a>, <a href="https://simonwillison.net/tags/john-resig">john-resig</a>, <a href="https://simonwillison.net/tags/jquery">jquery</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a></p>

2007-10-05T22:26:10+00:002007-10-05T22:26:10+00:00https://simonwillison.net/2007/Oct/5/yslow/#atom-tag

<p><strong><a href="http://developer.yahoo.com/yslow/netpanelpatch.html">YSlow: Bug (fix) in Firebug&#x27;s Net Panel</a></strong></p> The latest release of the YSlow page analysis plugin (announced at FOWA) also fixes a misleading bug in Firebug’s Net panel. <p>Tags: <a href="https://simonwillison.net/tags/firebug">firebug</a>, <a href="https://simonwillison.net/tags/firefox">firefox</a>, <a href="https://simonwillison.net/tags/fowa">fowa</a>, <a href="https://simonwillison.net/tags/profiling">profiling</a>, <a href="https://simonwillison.net/tags/steve-souders">steve-souders</a>, <a href="https://simonwillison.net/tags/yahoo">yahoo</a>, <a href="https://simonwillison.net/tags/yslow">yslow</a></p>