Program Testing Frameworks

Tooling
1

Program Testing Frameworks

Curious to hear people’s thoughts on program testing frameworks.

Testing a program is arguably the most trivial and cumbersome aspect of developing on Solana.

Everyone knows the pain of writing a program in Rust, then having to move to TypeScript to write tests with web3.js or anchor.

Anchor

If you’re using Anchor, a lot of the redundancy can be taken out by providing serialization and IDL interpreting, but there’s still the acts of:

  • Creating a test per instruction of your program
  • Generating Keypairs
  • Adding assertions

Anchor actually continues to creep closer to better testing with it’s new resolve-accounts feature, which attempts to use an IDL account’s seeds config to resolve a PDA from some inputs.

Other Tooling

In addition to Anchor, we’ve seen some tools that do a great handful of things that pertain to testing:

  • solana_program_test: The original Rust testing crate for unit tests on programs using cargo.
    • Lots of devs struggled to wrap their head around this crate, but it provided a way to write unit tests within your program’s crate, which is very convenient
  • solita: TypeScript types for all of your structs/enums from your program
    • Anchor did this first of course, but with Solita you actually get more TS-native type bindings and it can be a bit easier (or more intuitive) syntax for writing tests or client code on your program
  • solana-playground: Acheron’s powerful browser IDE
    • SolPG actually lays down a generalized test harness, where you just need to input the keys for any accounts
  • amman: Local validator harness and JS client-side code
    • Amman has a lof of what we need already to do testing from the JS/client side.
    • What else does it need?

Ideas

Some ideas proposed for testing automation:

  • Test Coverage metrics on programs
  • (Bokken) Local Validator Debugger
  • (Armani) Declarative-style testing framework
    • Similar to the “declare and deploy” nature of Helm or Terraform
  • (Sohrab) A test container you can submit a program to for automated unit testing
    • This is akin to functional containers for unit/integration tests or vulnerability scans, ie. AquaSec or SonarQube

Questions

Let’s try to answer the following questions:

  • What existing functionality do we want to see expanded on? (Anchor, Amman)
  • Should testing Solana programs be one big suite with pluggable components, or are we better off with a slew of libraries from various authors?
  • Would a testing container with declarative configs be something useful for development?
  • How much of this could be automated or boiled down to easy configs

Considerations:

  • :bulb: Telling the test suite your instructions are sequential
  • :bulb: Keypair generate (enough already!)
  • :bulb: Airdropping & funding test keypairs
  • :bulb: More comfortable use of local validator (Amman does this well)
2 Likes