DB Dilemmas : How to Define Requirements? Part 2


3 min read

DB Dilemmas : How to Define Requirements? Part 2
Play this article

Requirements Framework:

1. Query Patterns:

  • Types: What are the main types of queries? (e.g., CRUD, search, aggregation)

  • Frequency: How often are these queries expected to run?

  • Complexity: Are there any complex queries involving multiple joins, sub-queries, or advanced functions?

2. Performance and Scalability:

  • Read/Write Ratio: Estimate of read vs. write operations.

  • Expected Latency: Desired response times for different operations.

  • Growth: Projected data growth over time (e.g., per month, per year).

  • Concurrent Users: Estimate of the maximum number of simultaneous users.

3. Transaction Type

  • OLTP vs. OLAP: Operational vs. analytical processing requirements.

  • Hybrid Needs: Are there combined OLTP and OLAP requirements?

4. Availability and Fault Tolerance:

  • Downtime: Maximum allowable unplanned downtime.

  • Backup: Frequency and type (e.g., full, incremental).

  • Recovery: Expected time for data restoration.

5. Data Model and Flexibility:

  • Alterations: Anticipated changes to the data schema over time.

  • Schema Flexibility: How rigid or flexible is the data structure?

  • Null Values: Are null values allowed? If so, for which attributes?

6. Complexity:

  • Query Complexity: How intricate are the queries in terms of logic and joins?

  • ACID Compliance: Is there a need for Atomicity, Consistency, Isolation, Durability?

  • Referential Integrity: Are there interdependencies between data tables/entities?

7. Data Types and Structure:

  • Data Types: List of data types needed (e.g., strings, integers, date/time, blobs).

  • Joins: Frequency and complexity of join operations.

  • Normalization: Degree of data normalization vs. de-normalization.

8. Security:

  • Data Encryption: Requirements for encrypting data at rest and in transit.

  • Authentication: Type and level of authentication required.

  • Authorization: Levels of access control and granularity.

9. Operational:

  • Memory Usage: Anticipated in-memory data storage needs.

  • Pooling: Requirements for connection or thread pooling.

  • Maintenance: Frequency and type of maintenance operations.

10. Integration:

  • External Systems: List of external systems that the database needs to integrate with.

  • Data Import/Export: Requirements for data ingestion or extraction.

  • Data Warehousing: Compatibility with popular OLAP DB formats.

11. Compliance:

  • Regulatory Needs: Any specific regulatory compliances like GDPR, HIPAA, etc.

  • Audit: Requirements for auditing and logging data access and changes.

12. Budget:

  • Cost Constraints: Any budgetary constraints for setup, maintenance, or scaling.

13. Architecture and Data Propagation:

  • Event-Driven Requirements: Is there a need for an event-driven architecture? What are the event sources and sinks?

  • CDC Requirements: Is there a need to capture and propagate data changes in real-time or near-real-time? Which systems or databases are the sources for CDC?

  • CQRS: Is there a requirement for separate read and write data models? How will command and query models synchronize?

  • Event Sourcing: Is there a need to capture the state changes of entities as a sequence of events?

  • Integration Patterns: Are there specific integration patterns to be followed, like pub-sub, event streaming, or message queues?

14. Miscellaneous:

  • Vendor Lock-in: Acceptance level for being tied to a specific vendor or solution.

  • Geographical Distribution: Will the solution cater to a global audience, requiring data residency considerations or multi-region deployments?

  • Migration: Are there any existing data sources that need to be migrated?

Now let's Put it into action, how do all these things come together, let's decide DB for User Service in Part 3 : https://aayushgupta.tech/db-dilemmas-user-service-for-e-commerce-part-3