Seeing is Believing: A Tale of Two Demos (and the Power of Caching)

In this post, I want to show you the demo for the project after making the caching work. Below are two demos of the exact same analysis request. The only difference is that one is a cold start, and the other benefits from the now-functional cache.

Demo 1: The Cold Start (Without Cache)

This first demo shows what happens when the application analyzes a ticker for the very first time.

What’s happening behind the scenes:

1. The user requests an analysis for a new ticker.
2. The data layer connects to the SEC EDGAR database through HTTP requests.
3. It downloads multiple large filing documents (10-Ks and 10-Qs). This network operation is the source of the delay.
4. Once the data is downloaded, the AI analysis proceeds.
5. The filing data and LLM response data (in this case, OpenAI response) are saved to a local cache for future use.

Analysis without a pre-existing cache. The analysis roughly takes 40 seconds to complete.

Demo 2: The Warm Start (With Cache)

Now, let’s run the exact same analysis request a second time.

What’s happening now:

1. The user requests the same analysis again.
2. The data layer first checks the local cache directory we configured in the lifespan function.
3. Cache Hit! It finds the required filing documents then LLM response on the local disk.
4. The data is loaded directly from the disk, completely bypassing the slow internet download.
5. The AI analysis proceeds within 10 seconds.

The exact same analysis, but this time the data is served from the local cache.

The difference is quite exciting. The analysis is now nearly ‘instantaneous’.

PS: Never thought this would be the first Youtube video I’d ever made and posted online, but here we are as embedding an iframe in Hexo for blogs seems quite straightforward. I hope you enjoyed the demo and found it insightful. :)