Building a REST and real-time API with Express, Feathers and MongoDB
There are many ways to create RESTful APIs with NodeJS and Express but things get a little more complicated when you also want to add real-time functionality and expose the same or similar functionality using websockets. Setting up shared authentication, finding consistent event names and making sure events get sent to the right clients are only some of the challenges. There are of course full blown real-time frameworks like Meteor or Sails but if you like the more minimalistic and flexible approach of Express, Feathers might be an alternative worth considering.
Feathers is a very small wrapper over top of Express 4 so it is nearly a drop-in replacement and can be used in place of Express in almost any existing application. You can still use all the same middleware, but you also get real-time, RESTful services right out of the box.
In this post I’d like to show you how to create a todo service with Feathers and MongoDB as the database that provides a REST and websocket API. We will render the Todo application HTML with the initial data on the server using an Express view engine and then keep it up-to-date in real-time with SocketIO and jQuery. You can see it in action on this JSBin.
Feathers services
The important new concept Feathers adds to Express is that of services. A service is just a JavaScript object that provides one or more of the following methods and signatures:
As trivial as this might look, this simple abstraction is quite powerful. By disconnecting the functionality and data of your application from handling HTTP requests and responses or websocket events it becomes not just much more testable but also allows you to expose the same API through different protocols.
Our todo application will have a REST HTTP and SocketIO endpoint (other websocket libraries can be used through Primus). Feathers will also hook into the create, update, patch and remove methods and send events when they successfully return. An application then can be made real-time just by listening to those events and updating itself with the new data.
A simple service that keeps a list of Todos in memory, can retrieve them all and create new ones might look like this:
To get this service up and running we need to install feathers and the Express body-parser (so that the bodies of POST requests get parsed when someone wants to create a new todo via HTTP) with
npm install feathers feathers-rest feathers-socketio body-parser
The main application file (app.js) that sets up the REST and SocketIO providers and registers that todo service looks like this:
If you now run the file with
node app.js
And go to http://localhost:3030/todos you should see the JSON formatted data from the service. We can also create a new todo using the REST API by sending POST requests to that endpoint, e.g. from the command line using CURL:
curl 'http://localhost:3030/todos/' -H 'Content-Type: application/json' --data-binary '{ "text": "You have to do dishes!" }'I recommend using browser plugins to more easily work with the API. If you are using Chrome, for example, the JSON Formatter and Postman extensions.
A MongoDB service
Our Todo service example only stores its data in memory and only allows us to create new todos. To make it actually useful we probably want a persistent storage and the ability to modify and remove Todos. In short, we are looking for some form of CRUD functionality. If we wanted to use MongoDB for example we could now just go ahead and implement a service object that uses a NodeJS MongoDB client like MongoSkin or Mongoose or any other database access as our own service.
The good news is that MongoDB CRUD functionality already exists in the feathers-mongodb plugin which we might as well use. This post won’t go into more detail about it but if you are wondering how to customize that existing service, e.g. adding authentication or data processing, have a look at the plugin documentation. After installing MongoDB, starting it locally with the default settings and running
npm install feathers-mongodb
The entire todo server now looks like this:
This is all it takes with Feathers to set up a REST and SocketIO API that stores its data in MongoDB. Now running something like
curl 'http://localhost:3030/todos/' -H 'Content-Type: application/json' --data-binary '{ "text": "You have to do dishes!" }'Will create a new todo in the database and opening http://localhost:3000/todos/ will list all our stored todos where http://localhost:3000/todos/<id> will return a single todo based on its _id. You can also PUT to a specific todo to replace its data, PATCH to add or replace existing data and DELETE to remove it.
Rendering on the server
Now that the server and API is up and running we can focus on making an HTML page that shows our todos. To render the initial list on the server we will add EJS as the view engine for Express by running
npm install ejs
Keep in mind that besides the ability to use services our server is just a normal Express 4 application so you can use any view engine and middleware the Express ecosystem has to offer. To actually render the todos we register a middleware at the server root that does three things:
- Retrieve the registered service using app.service(‘todos’)
- Call service.find(params, callback) (with an empty object for no parameters)
- In the callback render the index view with the data returned from the service
The template will be a basic HTML page that loads jQuery and Twitter Bootstrap (to make things look nicer) from a CDN and connects to SocketIO. The file should be located in the /views folder (which is the default for Express views) and named index.ejs. In the template we iterate over all todos (that we assigned when calling res.render) using Array.forEach and render a list element with a checkbox (checked if the todo is completed), the todo text and a remove button. We also set the data-todo-id property to the todo _id so that we always know what todo is associated with a list element:
If you now go directly to http://localhost:3030/ you will see the list of all todos. The next and final step is to be able to add and remove todos and keep them up-to-date in real-time.
Real-time with jQuery
So far we talked about using the REST API and how to use a service internally to render a view. With app.configure(feathers.socketio()) in our application we are also exposing the same API via SocketIO however. In the <script> tag at the bottom of the index.ejs template we are already connecting to the socket so lets look at how to use that socket to access the todo service.
A service method can be accessed by emitting a `path::method` event with the normal service parameters. path is the registered service name without leading and trailing slashes (todos in our case). We an also listen to
`path methodEvent` events where methodEvent can be created, updated, patched and removed. Those events will be sent to every client every time any of those methods return successfully — no matter if it has been used through the socket or REST API or internally on the server. Feathers also provides a mechanism to filter which clients should get what event. Here are some examples to play around with:
Knowing how to use the service via sockets and being able to listen to events, we can now use jQuery to add new todos and update the list of todos. The jQuery events to listen to are:
- When submitting the .create-todo form, call todos::create
- When clicking the .delete button, call todos::remove
- When clicking the checkbox call todos::patch and set the complete property
The service events to listen to:
- On `todos created` add a new todo element to the list
- On `todos patched` update the checkbox for that todo
- On `todo removed` remove the todo list item
Below is the complete commented code that can be put in the <script> tag in index.ejs:
If you now restart the server and open http://localhost:3030/ you are able to add and remove todos and if you open the same page in a different browser window you will see the changes in real-time.
Conclusion
In this post we built a real-time todo application using Feathers that also offers a REST and SocketIO API with MongoDB as the backend. The todo data is initially rendered on the server using an Express view and then kept up to date with jQuery and Socket events. Of course, this can be easily adapted to your client MV* framework of choice. Especially if it already has data binding and a concept of models or services it is usually possible to simply replace it with a Feathers adapter to make your application update in real-time.
If you found this useful please share it with others and pass along your new found feathers knowledge. If you’re looking for more pro-tips for Feathers, check out more posts in the All About Feathers collection.
You can learn more about Feathers by checking out the guide and API docs, spelunking the code on Github or asking questions in our chat.
