How to Architect a Node.Js Project from Ground Up?

How to architect a Node.js project from ground up?

How to architect a Node.js project from ground up?

In this article, we will discuss how to architect a Node.js application properly, and why it is important. Also, we’ll look at what design decisions can lead us to in creating a successful digital product. Maybe you are building a new Node.js application from scratch. Perhaps you would like to refactor your existing application, or perhaps you want to explore Node.js application architecture and learn about the best practices and patterns. Whatever the reason, this article will help you.

Why should you read this post?

Well, it is true that there are many blog posts on the internet that cover this very subject. While there are some good articles on architecting Node.js projects, there are none that give you an in-depth explanation. Moreover, there are many blog posts that only elaborate on certain topics (i.e. layered architecture) but don’t tell you how everything fits together in an application. This is why I chose to write this article. I tried to research and compact all the information into one digestible piece so you don’t have to.

We will briefly go over how to architect a Node.js application properly and discuss the reasoning behind all the design decisions while building an actual dummy application.

We will discuss

  • Folder structure
  • Configuring environment variables
  • MVC pattern (Model, View, Controller)
  • Layered-architecture
  • Encapsulating Configurations

We will start with simple concepts and build on them. By the end of this article, you will be able to craft code that you are proud of.

Excited? Let’s get started!

Folder Structure

The organization is important while building large scale projects. We define our folder structure in a way so that it is easy and obvious to find code pieces later. As developers, we often collaborate with others. A well-defined code structure allows us to easily collaborate on a project.

Below is a sample folder structure that we have been using in my day job and it is working very well for us. We have delivered several successful projects with this structure.  We came up with this after many trials and errors. You are welcome to use this structure or modify it.

Sample folder structure

Sample folder structure

Alright, let’s build our first hello world API endpoint. As we build our sample application we will be populating these folders with code logic.

First, let’s take a look at our server.js file

Notice that we are requiring our app.js file. We will be writing all our app logic in app.js. It will be our main entry point for the app. Let’s take a quick look at the code.

For now, we’ve only added a route in our app.js. The main reason for separating these two files is to encapsulate logic. Let’s take a look at the npm script that I am using to run this application.

Please do make sure that you are able to run the application by doing npm run dev.

Let’s add resource routes

I bet you are eager to create some more routes. Let’s do that now. We will be creating the following files in our api/routes folder.

api/routes/authors.js

api/routes/books.js

Let’s just return some dummy JSON data from these routes.

You can do something similar for the author routes as well for now. Later in the post we will be discussing separation of concerns, and how we can architect our application with model view controller pattern. Before we do that, let’s cover one other important topic, setting up environment variables.

Configuring our environment variables

As programmers, we often underestimate the importance of organizing and configuring environment variables.  It is important that our apps work in various environments. This could be your colleagues’ computer, in a server, in a docker container, or in some other cloud provider. Therefore, setting up environment variables is crucial while architecting a Node.js application.

I am using dotenv library to manage environment variables in this application. First, I installed the library with npm i install dotenv --save. Then I created a  .envfile in the root directory. We add all of our environment variables in this .env file. Below is my sample .env setup.

It is a good practice to gather our variables from .env file and map them into well-named variables and export them through a module. Let’s create a file config/index.js.

The main reason for doing this is to manage our environment variables in one place. For some reason, we may decide to have multiple .env files. For instance, we may decide to have a separate .env for deployment with docker. We may also have other configuration variables. We would like to manage these variables efficiently that’s why we are following this convention.

Alright, now let’s see how we can import these variables into server.js

We have set up our environment variables. Let’s dive into the model-view-controller pattern now.

Model-View-Controller Pattern

MVC

MVC

Modern web applications are big and complex. To reduce complexity we use the Separation of responsibility principle (SRP). Using SRP ensures loose coupling, maintainability, and testability. MVC pattern embodies this philosophy of separation of responsibility. Let’s take a look at the different parts of MVC.

Model

Model components are responsible for application’s data domain. Model objects are responsible for storing, retrieving, and updating data from the database.

View: 

It is the user interface of our application. In most modern web applications, the view layer is usually replaced by another single page application, for example, a React.js or an Angular application.

Controllers:

They are responsible for handling user interaction. They interact with models to retrieve information and ultimately respond to user requests. In smaller applications, controllers can hold business logic. However, it is not good practice for larger application; we will look into a layered architecture later in this article to further elaborate on why this is.

Now, let’s take a look at how we can add this pattern to our application. I will be using mongodb as our database for this demo. I have created a new controller and a model to implement this pattern. First, let’s take a look at the author model.

We are defining our database-related schemas in the model as well. The controllers will deal with all the fetching and business logic for now. So let’s take a look at the controller.

Now we can slim down our router as follows:

Using this pattern separates our concerns and keeps the code clean, organized and testable. Our components are now following the single responsibility principle. For instance, our routes are only responsible for returning a response; controllers handle most of the business logic and models take care of the data layer.

Note: To get the code up to this point please check the following github repo:

https://github.com/Shadid12/starter_node/tree/part-2

Let’s say our business requirement has changed. Now, when we are adding a new author, we have to check if they have any best selling titles and whether the author is self-published or he/she belongs to a certain publication. So now if we start implementing this logic in our controllers things, begin to look rather messy.

Looks at the code below, for instance:

Now, this controller becomes responsible for doing multiple actions, this makes it harder to test, messy, and it is breaking the Single Responsibility Principle. 

How do we solve this problem? With the layered architecture!

Layered Architecture for Node.js

We want to apply the separation of concerns principle and move our business logic away from our controllers. We will create small service functions that will be called from our controllers. These services are responsible for doing one thing only, so in this way, our business logic is encapsulated. That way, if, in the future, requirements changes, we will only need to change certain service functions, and it will prevent any domino effects. With layered architecture, we build applications that are agile and allow changes to be introduced very easily when necessary. This architecture is also referred to as a 3-layer-architecture.

Here’s a visual breakdown of what we are about to do:

3-layer-architecture

3-layer-architecture

Alright so let’s break down our previous controller to use this architecture.  To start, we will need to create services to handle specific events.

Notice that service functions are designed to do one specific task. This way, our services are encapsulated, testable, and open to future changes without any major side effects.

Encapsulating Configurations

We write a fair amount of configuration code in our Node.js application. These usually run when the application boots up. It is good practice to have these encapsulated inside a function. This will allow us to track these files better and debug them if necessary.

Let’s elaborate on this with an example. Below we have our app.js file

We have a couple of things that are just configuration code. For instance, database connection, body parser, and cors setup are all server configuration code. We can move them into their own separate functions inside config folder.

And now we can use those functions in our app.js

And that’s it. Our app.js is now looking much cleaner.

Finally, here are the key points to keep in mind for a Node.js project architecture:

  1. Apply proper folder structure: It allows us to easily locate files and code. Also enables better collaboration with the team;
  2. Configuring environment variables: Configure and manage environment variables properly to avoid deployment;
  3. MVC pattern (Model, View, Controller): Apply MVC pattern to decouple, testable, and maintainable code;
  4. Layered Architecture: Apply layered architecture to separate your concerns. Use services extensively to encapsulate your business logic;
  5. Encapsulating Configurations: Separate configuration code from application logic.

We briefly went over the core concepts of Node.js project architecture. I hope this article was helpful to you and gave you some insights on how to architect your own project. I would love to hear what you think about this blog post. Please share your thoughts in the comment, if you enjoyed reading this please like and share. Until next time!

About the author

Stay Informed

It's important to keep up
with industry - subscribe!

Stay Informed

Looks good!
Please enter the correct name.
Please enter the correct email.
Looks good!

Related articles

Create simple POS with React.js, Node.js, and MongoDB #16: Order Screen

In this chapter, we are going to create an order page. The order page displays the products and the calculator to help calculate the total price. ...

Working With API in React Application using Axios and Fetch

We know that only API can separate the frontend from the backend. This tutorial will show the typical scenario of how to properly call Axios and ...

23.09.2020

Minimize Downtime by Creating a Health-check for Your NodeJS Application

When an API receives more traffic than it can handle, it will crash and be unavailable until someone manually brings it back online. This is referred ...

7 comments

Nikita Bragin December 19, 2019 at 10:06 am
1

Hello, Shadid, great article! Have you worked with Microservices architecture? Is it make sense to start building an app using Microservices from the very beginning?

 
Shadid Haque December 19, 2019 at 5:33 pm
0

Hi Nikita, great question. It really depends on the business strategy. If you are building a digital product focused on scalability, agile practices and fast growth I would recommend creating microservices from very beginning. If you are a startup focused on fast growth make it a monolith and break it up as you grow. 🙂

 
 
Nikita Bragin December 20, 2019 at 12:32 pm
0

Thank you, agree!

Melvin Carvalho April 14, 2020 at 3:08 pm
0

Great article

Just a note on plurals

At the top you had the folder “configs”

Later you say:

> We can move them into their own separate functions inside config folder

Do you have a view on the “S” at the end of config? I’ve always found this a difficult problem in programming because adding an S can change the stem

 
Nikhil Kapre May 26, 2020 at 12:44 pm
0

Great article Shahid! Once consideration one might have is to maintain configurations by environments (DEV, STAGE, PROD). While you specify a way to maintain it in the project itself, in projects, configurations are typically accessed from an external configuration repository. As a feedback, one very important thing you might want to add to enhance your article is to also lay out the structure for tests.

 
 
Prashanth Prabhu June 4, 2020 at 6:55 am
0

I agree with Nikhil

Purnima Das August 11, 2020 at 9:24 am
0

Hello Shadid, thank you for posting this article, I am new to node js and I have few queries: after quickly going through this article, I understand that this example uses only node.js. What if I have to develop application that uses node.js and sails.js in order to build endpoint APIs, and I also want to include react.js in order to create UI, so what would you suggest and why .. should I have node js, sails js and react js all under one project folder or should I split it into 2 i.e. one for UI and the other for endpoints API?

Thank you,
Purnima Das

Sign in

Forgot password?

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy

Password recovery

You can also try to

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy