This is our first blog post. We will post weekly updates every Friday. Without further ado, let's jump into what's happening at Theta Data.
Processing 10 years of options data
Uncompressed, this data is a whopping 4 petabytes. Yes, that's 4 million gigabytes. Our data center operations (DC OPs) team built 4 nodes to process the data. These nodes process a combined 16 million quotes / trades per second. We are averaging around 2 months of historical data added to our database every day.
Node specs: AMD 5950x (16-core), 64GB of RAM, 500GB NVME SSD.
Free Data
We are working diligently to add free End of day (EOD) data. EOD data contains the open, high, low, close, trade prices along with the total amount of trades (count) and the total contracts traded (volume).
What's happening next week?
We will add support for all the request types listed here. But wait, there's more... We will also have 2 years of data recorded by next Friday.
A Cool Fact
By the time we're finished recording the 10 years of data, the database will have over 10 billion entries. Each entry is a day of data for a certain contract, that equates to 27+ million years of options data.
Thanks for tuning in, check back in a week for more!
- Bailey Danseglio CEO @ Theta Data