Data Store Project Charter

Scoping Outline:

Challenges:

• What: "Create ways to..." / "Redesign the..."
  • Design a data governance process that does not require general users (often junior developers) to actively comply with and knowingly implement team-wide policy.
  • Create a cohesive way for internal API consumers to specify their access, organization and persistence policies for application/experiment data.
• For Whom: "For <user>... (considering <other stakeholders>)..."
  • For developers of our platform, considering that future contributors or peers seeking to replicate our work need forward-compatible data sufficient for re-evaluation post hoc.
  • ...And also considering that our own contributors are often junior developers, whose responsibility is to implement working application code on their individual projects, not to bear the burdens of policy compliance.
• Context: "In a world where..." / "Keeping in mind that..."
  • Keeping in mind that observations of student behavior - and generative/derived content delivered to students - are not repeatable/recoverable if unsaved, such that we need a storage method which always preserves original observations and their context/provenance.
  • Keeping in mind that future contributors are often facing a heavy cognitive load when learning our codebase.

Considerations:

• Goals: "We aim to..."
  • Provide storage that supports arbitrary data manifestation and projection.
  • Provide storage that can reconstruct/retrieve/replay all observations and their influence on manifestations/projections; storage is lossless.
  • Provide storage that is inseparable; all persistent manifestations are packaged within one cohesive container, such that clients of the data store do not need to micromanage where their data is persisted - the application bundles all data related to a run and the storage module is responsible for ensuring persistent representations are available to API clients.
  • Provide storage that supports first-class representation of experimental/software version provenance and other metadata (extensible by, but not the responsibility of, API clients).
  • Provide a storage API which is consistent across developer/client API needs, yet supple and accommodating to diverse record types, manifestations and projection strategies.
• Crux: "We really need to..."
  • Support reproducibility and total recall as a first-class feature/responsibility of the data store service.
  • Make data provenance and append-only features invisible to API consumers whose only need is to access a projected storage interface.

Framing Checklist:

We should discuss these qualities:

• You tried on some behaviors (like empathy and rapid prototyping) on a current project before launching a new one.
• Project is a human, subjective challenge (understanding people is key to the project success).
• Project is geared toward discovery (not optimization).
• Challenge can be solved with a product, service, or event, not a strategy or system (for your first project).
• Those most affected by the work are acknowledged as actors with agency, not simply receivers of outcomes.
• Framing doesn’t embed a solution.
• Framing doesn’t (unintentionally) assume the form of the solution.
• Framing doesn’t (unintentionally) presume people’s needs.
• Goal of the project work is clear without dictating the specific solution outcome.
• You and your team actually care about this challenge. (If not, why are you doing it?)