Node and express basics

Use case

Use case

express is a library for backend development

fundamentally, it can receive HTTP requests and send corresponding responses

typical use cases: developing a website (responses are HTML), developing an API (responses are JSON or another data format)

Use case

examples:

receives a request for some API data (e.g. /api/article/19e2), sends back that data

receives a request for a static resource, sends back that resource (e.g. styles.css, logo.png, ...)

receives a request for a page (e.g. /news), renders and sends back that page as HTML

Libraries

Libraries

• plain node (createServer)
• connect (includes middleware)
• express (includes middleware, routing, view rendering, ...)

most projects will use express

Hello world

Hello world

hello world with express:

// server.js
import express from 'express';

const app = express();

// provide a function that handles a get request to "/"
// and sends a response
app.get('/', (req, res) => {
  // note: we should actually return a complete HTML file
  res.send('<h1>Hello World!</h1>\n');
});

// listen on localhost:3000
app.listen(3000);

Hello world

hello world without express (see https://nodejs.org/en/docs/guides/getting-started-guide/):

// server.js
import http from 'http';

const handler = (req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/html; charset=UTF-8');
  res.end('<h1>Hello World!</h1>\n');
};

const server = http.createServer(handler);
server.listen(3000);

Starting the server

basic start script to start the server:

{
  "scripts": {
    "start": "node server.js"
  }
}

Starting the server

to start a server and restart it every time some code changes:

install nodemon from npm and specify a dev script in package.json:

{
  "scripts": {
    "start": "node server.js",
    "dev": "nodemon server.js"
  }
}

Request and response

Request and response

web development in node is based on request handler functions, e.g.:

(req, res) => {
  res.send('<h1>Hello World!</h1>\n');
};

Request and response

a request handler function receives two arguments:

• req - represents the incoming request
• res - represents the response that we will send

The request object

example for a request object:

{
  "method": "GET",
  "path": "/products/123",
  "params": { "id": "123" },
  "headers": { "user-agent": "Mozilla/5.0 (Windows ..." }
}

class in plain node: IncomingMessage

class in express: Request

The response object

class in plain node / connect: ServerResponse

subclass in express: Response

Examples

Examples

send a hello world message under /hello:

app.get('/hello', (req, res) => {
  res.send('<h1>Hello World!</h1>\n');
});

Examples

send a static file:

app.get('/favicon.ico', (req, res) => {
  res.sendFile('public/favicon.ico');
});

Examples

get database entries and send them back as JSON:

app.get('/api/products', async (req, res) => {
  // use a DB library for accessing the database
  const sqlQuery = 'SELECT * FROM product';
  const [results] = await db.query(sqlQuery);

  // send the results as JSON
  res.json(results);
});

Examples

get a database entry and send it as JSON based on a URL parameter:

app.get('/api/products/:id', async (req, res) => {
  const id = Number(req.params.id);

  const sqlQuery = 'SELECT * FROM product WHERE id = ?';
  const [results] = await db.query(sqlQuery, [id]);

  res.json(results[0]);
});

Examples

get a database entry and send it as JSON based on query parameters

example request URL: /api/products?min_rating=4&max_price=200

app.get('/api/products', async (req, res) => {
  const minRating = Number(req.query.min_rating);
  const maxPrice = Number(req.query.max_price);

  const sqlQuery =
    'SELECT * FROM product WHERE rating >= ? AND price <= ?';
  const sqlParams = [minRating, maxPrice];
  const [results] = await db.query(sqlQuery, sqlParams);

  res.json(results);
});

Examples

render some HTML based on a database entry

app.get('/products/:id', async (req, res) => {
  const sqlQuery = 'SELECT * FROM product WHERE id = ?';
  const sqlParams = [req.params.id];
  const [results] = await db.query(sqlQuery, sqlParams);

  // look for the "product_detail" template and render it based on
  // the retrieved data
  res.render('product_detail', {
    name: results[0].name,
    price: results[0].price,
  });
});

Topics overview

Topics overview

• common topics
  • basic routing
  • sending responses: content, status, headers
  • route parameters and query parameters
  • middleware
• API topics
  • sending and receiving JSON data
  • same origin policy
  • Rest APIs
  • testing APIs
• HTML topics
  • sending files and serving static folders
  • rendering HTML
  • handling form submissions

Basic routing

Basic routing

app.get('/', (req, res) => {
  res.send('<h1>Home</h1>\n');
});
app.get('/about', (req, res) => {
  res.send('<h1>About</h1>\n');
});

other HTTP methods are handled via: .post, .put, .delete, ...

Sending responses

Sending responses

methods of response objects:

• .status() - for setting the status code
• .set() - for setting headers
• .send() - for sending content

Sending responses

explicit example:

res.set({ 'Content-Type': 'text/html' });
res.status(200);
res.send('<!DOCTYPE html><html>...</html>');

Sending responses

by default, the Content-Type will be text/html and the status will be 200

shorter version:

res.send('<!DOCTYPE html><html>...</html>');

Sending responses

another example:

res.status(404);
res.set({ 'Content-Type': 'text/plain' });
res.send('Document not found.\n');

Sending responses

response objects support method chaining:

res
  .status(404)
  .set({ 'Content-Type': 'text/plain' })
  .send('Document not found.\n');

URL parameters

URL parameters

examples of URLs with parameters:

• /products/123 (route parameter / path parameter)
• /products/?min_rating=4&max_price=200 (query parameters)

Route parameters / path parameters

example URL: /products/123 (references a specific product by an ID)

app.get('/products/:id', (req, res) => {
  const productId = req.params.id;
  // ...
});

Query parameters

example URL: /products?min_rating=4&max_price=200 (filters by some criteria)

app.get('/products', (req, res) => {
  const minRating = Number(req.query.min_rating);
  const maxPrice = Number(req.query.max_price);
  // ...
});

Middleware

Middleware

middleware can respond to requests or modify reqest / response objects

Middleware

example use:

import compression from 'compression';
import express from 'express';
import morgan from 'morgan';

const app = express();

// log all incoming requests
app.use(morgan('common'));
// serve content of the "public" folder as-is if found
app.use(express.static('public'));
// compress all responses
app.use(compression());

app.get('/', (req, res) => {
  // ...
});

Middleware and routing

// log all incoming requests
app.use(morgan('common'));
// for any request that starts with /api, parse its
// JSON body if it exists
app.use('/api', express.json());
// compress API responses
app.use('/api', compression());
// requests to /assets/...: send back static files
app.use('/assets', express.static('public'));

Example middleware

• express.json, express.urlencoded, ... : parse the request body and make it available as req.body
• cookie-parser: parses cookies and makes them available under req.cookies
• compression: compresses the response content
• express.static: sends static files (e.g. index.html) if available
• express-session: stores session data (available under req.session)
• express-openid-connect or passport: user authentication
• morgan: logging
• ... (see: list of available express middleware)

Sending and receiving JSON data

Sending and receiving JSON data

sending JSON (manually):

res.setHeader({"Content-Type", "application/json"});
res.send(JSON.stringify(data));

with the .json() helper:

res.json(data);

Sending and receiving JSON data

receiving (parsing) a JSON body via middleware:

app.use("/api", express.json());

data will be available as req.body

Same-origin policy and CORS

Same-origin policy

Same-origin plicy: safety rule in the browser

by default, a webpage on one site (e.g. www.example.com) is not allowed to make requests to another site (e.g. api.example.com)

Same-origin policy

example: go to one website (e.g. Wikipedia), open the browser's JavaScript console and request another website, e.g. via fetch("https://google.com")

result: the request is prohibited

reason: to prevent exploitation of cookie-based authentication

Cross-Origin resource sharing

The requested site may allow Cross-Origin resource sharing (CORS) for some URLs or for all URLs

This is done via the HTTP header "Access-Control-Allow-Origin"

example: the jsonplaceholder API enables CORS for all sites - so fetch("https://jsonplaceholder.typicode.com/todos") works from any site

CORS with express

enabling CORS in express:

import cors from 'cors';

allowing requests from all domains:

app.use('/api', cors());

allowing requests from a specific domain:

app.use('/api', cors({ origin: 'https://example.com' }));

Rest APIs

Rest APIs

We'll create an example Rest API with express and a database that's connected via the mysql2 library.

We will create a database named "tasklist" with a single table named "task"

Rest APIs

note: Different Rest APIs can behave differently on mutations

example: possible responses when an entry has been created via POST:

• return the complete new entry in the response body (including its new id)
• return just the new id in the response body
• return no response body

Rest APIs

basic setup:

app.use('/api', express.json());
app.use('/api', cors());

Rest APIs

get all entries from a resource:

app.get('/api/tasks', async (req, res) => {
  const [results] = await db.query('SELECT * FROM task;');
  res.json(results);
});

Rest APIs

get by a specific id:

app.get('/api/tasks/:id', async (req, res) => {
  const id = Number(req.params.id);
  const sqlStatement = 'SELECT * FROM task WHERE id = ?';
  const [results] = await db.query(sqlStatement, [id]);
  if (results.length === 0) {
    // 404 Not Found
    res.status(404).send();
  } else {
    res.json(results[0]);
  }
});

Rest APIs

get by some query parameters (is_done, due_date):

app.get('/api/tasks/', async (req, res) => {
  // array of conditions - e.g. ['is_done = ?', 'due_date = ?']
  const conditions = [];
  // array of corresponding values - e.g. [1, '2020-10-30']
  const values = [];

  if (req.query.due_date !== undefined) {
    conditions.push('due_date = ?');
    values.push(req.query.due_date);
  }

  if (req.query.is_done !== undefined) {
    conditions.push('is_done = ?');
    // can be either "0" or "1"
    values.push(Number(req.query.is_done));
  }

  let sqlStatement = 'SELECT * FROM task';
  if (conditions.length > 0) {
    sqlStatement += ` WHERE ${conditions.join(' AND ')}`;
  }

  // sqlStatement = complete SQL statement with variables, e.g.:
  // SELECT * FROM task WHERE is_done = ? AND due_date = ?
  // values = corresponding values, e.g.:
  // [1, '2020-10-30']

  const [results] = await db.query(sqlStatement, values);
  res.json(results);
});

Rest APIs

creating a new entry:

app.post('/api/tasks', async (req, res) => {
  const isDone = req.body.is_done;
  const dueDate = req.body.due_date;
  if (isDone === undefined || dueDate === undefined) {
    // 400 Bad Request
    res.status(400).send();
    return;
  }
  const sqlStatement = `INSERT INTO task (due_date, is_done) VALUES (?, ?)`;
  const values = [dueDate, isDone];

  const [results] = await db.query(sqlStatement, values);

  // 201 Created (or 200 OK)
  res.status(201).json({ id: results.insertId });
});

Rest API

replacing an entry - via put:

app.put('/api/tasks/:id', async (req, res) => {
  const isDone = req.body.is_done;
  const dueDate = req.body.due_date;

  if (isDone === undefined || dueDate === undefined) {
    // 400 Bad Request
    res.status(400).send();
    return;
  }
  const id = Number(req.params.id);
  const sqlStatement = `
    UPDATE task
    SET due_date = ?, is_done = ?
    WHERE id = ?
  `;
  const values = [dueDate, isDone, id];
  const [results] = await db.query(sqlStatement, values);
  if (results.affectedRows === 0) {
    // 404 Not Found
    res.status(404).send();
  } else {
    res.send();
  }
});

Rest API

delete an entry:

app.delete('/api/tasks/:id', async (req, res) => {
  const id = Number(req.params.id);
  const sqlStatement = `DELETE FROM task WHERE id = ?`;
  const values = [id];
  const [results] = await db.query(sqlStatement, values);

  if (results.affectedRows === 0) {
    // 404 Not Found
    res.status(404).send();
  } else {
    res.send();
  }
});

Testing APIs

Testing APIs

APIs are not as strightforward to test as scripts or UIs

ways to test an API:

• use an application like postman to send HTTP requests and inspect the responses
• write automated tests

Automated tests

simple test without any extra tools:

async function testGetTodos() {
  console.log('/todos: returns an array of 200 todos');
  const res = await fetch(
    'https://jsonplaceholder.typicode.com/todos'
  );
  const data = await res.json();
  if (data.length !== 200) {
    console.log('FAILED');
  }
}
await testGetTodos();

Automated tests

test with node's built-in testing tools:

import { test } from 'node:test';
import assert from 'node:assert';

test('/todos: returns an array of 200 todos', async () => {
  const res = await fetch(
    'https://jsonplaceholder.typicode.com/todos'
  );
  const data = await res.json();
  assert.strictEqual(data.length, 200);
});

Sending files and serving static folders

Sending files and serving static folders

sending just a single static file:

app.get('/favicon.ico', (req, res) => {
  res.sendFile('public/favicon.ico');
});

Serving static folders

specifying a folder that contains static files via middleware:

look for static content for every request:

app.use(express.static('public'));

look for static content only if the URL starts with /static:

app.use('/static', express.static('public'));

Rendering HTML

Rendering HTML

express can generate HTML with dynamic content - the HTML will be generated (rendered) for every request

Rendering HTML

manually (can be open to attacks):

app.get('/', (req, res) => {
  const name = 'world';
  res.send(`
    <!DOCTYPE html>
    <html>
      <head><title>Hello ${name}</title></head>
      <body>Hello ${name}</body>
    </html>
  `);
});

Rendering HTML

via a template engine:

• ejs: website, express integration
• handlebars: website, express integration
• pug: website, express integration

Rendering HTML

general procedure:

import express from 'express';
import myengine from 'myengine';

const app = express();

// specify the rendering engine
app.set('view engine', 'myengine');

app.get('/', (req, res) => {
  const name = 'world';
  // renders 'views/index.myengine'
  res.render('index', { name: name });
});

Rendering HTML

example with handlebars:

npm install hbs

app.set('view engine', 'hbs');

// ...

app.get('/', (req, res) => {
  const name = 'world';
  // renders the file 'views/index.hbs'
  res.render('index', { name: name });
});

(replace hbs with ejs / pug for other template engines)

Exercises

exercises:

• create a website with different pages (home, about, newsletter, ...)
• create a website that displays information from a public API (e.g. https://pokeapi.co/)
• create a website that displays information from a database on your computer

Exercises

Pokeapi part 1:

app.get('/pokemon/:id', async (req, res) => {
  const id = req.params.id;
  const dataRes = await fetch(
    `https://pokeapi.co/api/v2/pokemon/${id}`
  );
  const data = await dataRes.json();
  res.render('pokemon', {
    id: id,
    name: data.name,
    imgSrc: data.sprites.front_default,
  });
});
app.get('/pokemon', (req, res) => {
  res.redirect('/pokemon/1');
});

Exercise: Pokeapi

Pokeapi part 2:

views/pokemon.hbs:

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>{{name}}</title>
  </head>
  <body>
    <h1>{{name}}</h1>
    <img src="{{imgSrc}}" />
  </body>
</html>

Forms

Request parameters

By default, the browser sends form contents in URL-encoded format, e.g.:

foo=1&bar=2&baz=3

in get requests: as part of the URL, e.g. https://google.com/search?ei=xyzg&q=foo...

in post requests: in the request body

Getting request parameters

in a get request: read req.query

in a post request: use express.urlencoded middleware, read req.body