Archival storage for Solana has historically expensive and centralized from the technology perspective; at the moment, BigTable tends to be the only reasonable choice for RPCs to store historical information back to genesis.
See this RFP on the development of technologies to enable alternative storage technologies and providers to provide low-cost, high-efficiency access to Solana archival data
Logistics
Take note of the end date (8/13) and be sure to make sure all criteria is met prior to sending in an application. The listed grant amount is a maximum allocation and is issued in USD-equivalent locked SOL and gated behind delivery milestones.
Ground Rules
This thread can be used for comments, questions, praise, and / or criticism, and is intended to be an open forum for any prospective responders. This thread is also an experiment in increasing the transparency through which RFPs are fielded by the Solana ecosystem too, so please be mindful that we’re all here to learn and grow.
Responses to this RFP are not required to be public, but if it is helpful to share notes or combine forces, then please use this thread for such purposes.
Is there some publicly available documentation of the often cited method of using Filecoin for this storage? I see it consistently mentioned by aeyakovenko:
Would love to see what the pros and cons of this approach have been so far. Based on my knowledge of Filecoin, the cost might be quite prohibitive.
I am working on this from the filecoin side along with folks from Triton. Triton just released https://old-faithful.net/ which has more details on how data is being onboarded to filecoin. Happy to answer any follow up questions!
To the point about cost being prohibitive, Filecoin is actually the cheapest option today. See this from messari:
Solution should provide relevant connection logic for the Solana RPC client
Solution must prove equivalence to the Solana ledger as determined by random-sampling of RPC calls
Does this mean that the solution must include a separate RPC that runs on a subset of the data? (eg. an epoch)
and
A complete security audit must be completed prior to production launch.
who is responsible for this? if the submitter, should this be factored as a cost? (problematic as it’s an unknown)
Does this mean that the solution must include a separate RPC that runs on a subset of the data? (eg. an epoch)
no, you can use the existing Solana RPC code. today, that RPC will pack things into BigTable / serve archival requests out of BigTable. the proposed solution needs to plug into that existing code to serve as a suitable replacement, and needs to store data from genesis to tip.
who is responsible for this? if the submitter, should this be factored as a cost? (problematic as it’s an unknown)
good point. don’t have a perfect answer for you since this component isn’t security-critical enough to require an audit; probably okay to waive this concern for the time being