Challenge: Leadspotr COSI Analysis

Now that you've seen the various architectural patterns and storage options in more detail, take another look at the COSI sheet you filled in at the end of the first module of this course. Is there anything you'd like to update? Have you considered the best practices for each of the different storage architectures? Edit the worksheet accordingly.

Let me know in the comments if there's anything in particular that you notice you approach differently after following this module.

For convenience, I've included the worksheet PDF as a download in this lesson.


Joris Jansen

Previously:
# COMMUNICATION
- HTTP (API Rest)
- A Mail Client (SES)
- DB Connection(s)

# ORGANISATION
- Client-Server (Frontend & Backend)
- The Backend can be a Monolith, but depending on my last 'point' of the Event-Driven approach, it could be service-oriented as well.
- The Backend 'code' can be organized in a Hexagonal or Multi-Layer Architecture
- You can implement an Event-Driven approach for finishing a Quiz. So that multiple processes can react based on those events such as; generating PDFs, sending emails, generating statistics, etc (RabbitMQ/Kafka).

# STORAGE
- SQL Database (Postgres)
- Object storage for generated PDF files (S3)
- Store ongoing quizzes on-device, for easy and fast back-and-forth between questions.

# IMPLEMENTATION
- Docker
- Managed Database Provider (RDS)
- Run the Application on the Cloud
- Leverage Serverless at the beginning (Lambda)
- CI/CD (GitHub Actions probably)

Now:
# COMMUNICATION
- HTTP (API Rest)
- Email (Email Client)

# ORGANISATION
- Client-Server (Frontend & Backend)
- Monorepo
- Backend can we written as Hexagonal/Onion Architecture

# STORAGE
- SQL Database
- Object storage

# IMPLEMENTATION
- Docker
- Managed Database Provider
- Cloud
- Serverless
- Deployment Pipeline (Github Actions)

Most of the things I changed are unnecessary implementations of rather complex things for such a 'small' application. If the Application is running Serverless already, generating a PDF or sending an Email within that same Lambda function is fine (compared to doing that in separate functions). Therefore you don't have to start new functions. implement some Event-Driven architecture, etc, etc.

REPLY
Andreas [ArjanCodes Team]

Nice idea! How would you tackle if we want to expand the application with additional functionality and more?

REPLY
Joris Jansen

When it comes to IMPLEMENTATION, you can either migrate to a managed Kubernetes/Cluster solution (ECS on AWS for example) or you could decide to keep things Serverless.
In both cases, depending on how many users you have and what functionality you want to add, you can decide to add this new functionality as a Microservice that communicates through Events.
If it makes sense, you could also add this functionality to the existing code, as long as the User Experience regarding the Request/Response cycle isn't affected by this.

REPLY
Andreas [ArjanCodes Team]

That is a very elegant solution! Thanks for sharing! :D

REPLY
Jake Atwell

1. I was thinking of using a nosql database for storing the quiz answers that users select. This could make sense because this would be data that could be written a lot. (We could potentially do a write each time the users selects an answer instead of waiting until they submit the whole form to save data in case the quiz is longer). This could avoid any bottlenecks of using a sql database for that purpose (which can only do one write at a time if I'm not mistaken), but I still think it makes sense to use a sql database to store the questions and possible answers because it's relational data. The user-selected quiz answer data could also be orders of magnitude larger than the question/answer data depending on the amount of people that actually take a quiz, whereas the question and possible answer data does not depend on the number of people taking the quiz. It seems a little weird to store data that's so similar in two different databases, but I don't see any other issue with it other than I'm not used to the idea. Am I missing any potential drawbacks?

2. I would use a cache for the quiz questions and answer choices.

3. I could use a pipeline architecture for doing the analysis of the user answer choices.

REPLY
Marcel Kerkveld

In my opinion, the answers are still very structured data, so to me a NoSQL does not make sense. In my experience, the serious SQL databases systems are quick enough to perform the transaction without being a bottle neck. I think you really need millions of concurrent users/migrations to warrant a different system and even then, you could do some caching/task queuing to manage peak usage. But that's just my opinion as a fellow course student ;)

REPLY
Andreas [ArjanCodes Team]

Are you considering having both a NoSQL and an SQL database? I would recommend not managing two different database systems can increase operational complexity and overhead. However, if time and resources are available, then of course it could be valid.

A good rule of thumb is to not store data that can be calculated from other data, this can become more intensive in terms of computation. But then we can use caching or views to speed up that process.

In general, I would say that this is a good plan!

REPLY