Building Cloud Services: Intro to Servlets, Spring Controllers and Spring Boot

2020, Jul 23
Link to project here.

In this blog we'll look into the basics such as what Servlets and Spring Controllers are, and the basic structure of a Spring Boot application.

Table Of Contents

Servlets

The technology that Java uses to handle incoming requests is typically Servlets. Servlets are just Java objects that contain methods to handle HTTP requests such as doGet(), doPost() and so on. When a web browser sends a HTTP request, this request often gets routed to a Web Container that contains one or more Servlets. The web container then uses a routing function that decides which servlet to forward the request to based on a web.xml file.

A First Cloud Service Servlet

Lets assume we are creating a video hosting service, here we have a simple example using a primitive servlet. We are simply storing a list of Video objects that contain the video's name, duration and URL.

public class VideoServlet extends HttpServlet {

    private List<Video> videos = new ArrayList<Video>();

    ...
}

Inside this class, we are overriding the doGet() and doPost() methods so that we can implement our business logic. With the doGet(), we are simply looping through the list of videos and sending a plain text string with the data back to the client.

@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp)
        throws ServletException, IOException {
    // Set MIME Type
    resp.setContentType("text/plain");

    // Loop through all the stored videos and print them for the client to see
    PrintWrite sendToClient = resp.getWriter();
    for(Video v: this.videos) {
        sendToClient.write(v.getName() + ": " + v.getUrl() + "\n");
    }
}

As we can see, this method is pretty simple, though the same can't be said for the doPost() method. Here we extract the data that is sent in the POST body, parse it, and then add it to the list.

protected void doPost(HttpServletRequest req, HttpServletResponse resp)
        throws ServletException, IOException {
    // Data Extraction
    String name = req.getParameter("name");
    String url = req.getParameter("url");
    String durationStr = req.getParameter("duration");

    resp.setContentType("text/plain");

    // Error checking
    long duration = -1
    try {
      // Data type conversion
      duration = long.parseLong(durationStr);
    } catch(NumberFormatException e) {
      // Invalid Number
    }

    // More Error checking
    if(name==null || url==null || durationStr==null ||
            name.trim().length < 1 || url.trim().length < 10 || duration <= 0) {
        resp.sendError(400);
        resp.getWriter().write("Missing ['name', 'duration', 'url]");
    } else {
        Video v = new Video(name, url, duration);
        // ** Logic Start **
        videos.add(v);
        // ** Logic End **
        resp.getWriter().write("Video added");
    }
}

Even with such a simple post request, using this primitive servlet means that there is a lot of boilerplate code that has to take place in order to run the single line of logic that adds data to the list.

web.xml

In order for the web container to know which servlet to send a request to, we need to configure the web.xml file.

<web-app xmlns="http;//java.sun.com/xml/ns/j2ee" version="2.4" 
      xmlns:xsi="http://www.w3.org/2001/XMLScheme-instance"
      xsi:schemaLocation="http:/java.sun.com/dtd..._2_3.dtd">
  <!-- Define Servlet -->
  <servlet>
    <servlet-name>video</servlet-name>
    <servlet-class>org.mobilecloud.VideoServlet</servlet-class>
  </servlet>

  <!-- Define Routing -->
  <servlet-mapping>
    <servlet-name>video</servlet-name>
    <url-pattern>/video</url-pattern>
  </servlet-mapping>
</web-app>

This is the traditional way of creating a web service. There are other modern methods allow us to do this routing in Java itself that allows us to maintain type safety and prevent mistakes that could occur when using xml format.

Spring Dispatcher Servlet

The Spring framework provides a specialized servlet called a Dispatcher Servlet. It allows us to stop writing boilerplate code to extract parameters and just write standard methods.

After a web container has routed the request to the servlet, it can perform a second layer of routing to an appropriate controller as defined in an xml file or via Java annotations. This routing is to individual methods inside a controller. So rather than having multiple Servlets with single doGet() methods, we can have methods with arbitrary names that the dispatcher can route to based on the URL or other conditions, providing much more routing flexibility.

Additionally, when a request path is routed to a particular method and before the method is invoked, it will look at the parameters that the method requires and figure out if it can extract said parameter from the HTTP request automatically.

Spring Controllers

Spring Controllers are just standard Java objects that are connected to a dispatcher servlet. Below is an example of a simple Spring controller:

@Controller
public class ContactsController {
    @RequestMapping("/contacts")
    public Contacts getContacts() {
        // ... Retrieve contacts...
        Contacts c = ...
        return c;
    }
}

Lets say that ContactsController was created without any annotations and only with the required business logic to perform its function. In order to connect it to the dispatcher servlet, all we need to do is add the @Controller annotation to let Spring know that we want this class to serve as a controller, and then the @RequestMapping("/contacts") annotation to let the servlet know we want requests to the /contacts path to be routed to it. If we wanted to extend the class, all we would need to do is to add the method we need along with an additional @RequestMapping() annotation with a path that is different from the first.

@RequestMapping("/friends")
public Contacts getFriends() {
    // ... Retrieve friends...
    Contacts f = ...
    return f;
}

Accepting Client Data with RequestParam Annotations

Now we want to allow users to search for contacts given some conditions, so they would need to send said conditions as request parameters for us to process. @RequestParam() is an annotation that can be attached to individual method parameters to inform the dispatcher servlet about them. 

@RequestMapping("/search")
public Contacts search(
        @RequestParam("search") String searchStr,
        @RequestParam("flag") int searchFlag) {
    // ... Search ...
    return c
}

The dispatcher will then automatically look for an HTTP request parameter with the key given in @RequestParam("flag") and map it to searchFlag. During this process, the dispatcher will also take the string value in the HTTP request, look at the type in the method and automatically convert it an int before passing to the method.

Accepting Client Data with PathVariable Annotations

Data that's sent from the clients to the server, doesn't have to just be embedded in the HTTP request frames, often we see links where the link itself has some semantic meaning to it.

Let's say rather than embed our search strings into our HTTP request parameters like /search?search=ab, we now want to instead be able to have links that look like /search/ab so that it is more clean looking easily sharable. We can do this by modifying the path and using the the @PathVar() annotation. The dispatcher will then extract someStr from the path and then pass it to the method.

@RequestMapping("/search/{someStr}")
public Contacts search(
        @PathVar("someStr") String searchStr,
        @RequestParam("flag") int searchFlag) {
    // ... Search ...
    return c
}

We can also mix and match the use of path variables and request parameters as we see fit, and this helpfully separates our business logic from the data extraction process.

Accepting Client Data with Request Body Annotations & JSON

The dispatcher servlet can pass much more sophisticated information than just simply extracting parameters from the HTTP request and dumping them into method parameters. It can actually extract entire custom Java objects that have been instantiated from your own classes.

Let's say we want a more sophisticated search, and created a Search class that would specify the parameters of the search and hold all the required data.

public class Search {
    private String first;
    private String last;

    // ...Getters/Setters...
}

We can then modify our search method to use this class directly using @RequestBody rather than having to take individual parameters and them construct the object inside the method.

@RequestMapping("/search/")
public Contacts search(
        @RequestBody Search s) {
    // ... Search ...
    return c
}

The annotation tells the dispatcher to take all the parameters in the request body and automatically convert them to an object of type Search before passing the method parameter s. There are various pre-configured HTTP Message Converters to handle these kinds of operations.

Accepting and Handling Multipart Data

For large sets of binary data such as video files, the methods we have discussed so far would be unsuitable. Instead we need to use a multipart request, so the server also needs to understand how to accept multipart data.

Say we are once again working on a video service where we can upload large videos. We create the method for receiving the video where we use the MultipartFile class, which streams the received data directly into storage rather than holding the entire file in memory until the upload is complete. If we wanted to receive multiple parameters, we could also make it an array of MultipartFiles.

@Controller
public class VideoService {
    @PostMapping("/")
    public boolean uploadVideo(
            @RequestParam("data") MultipartFile videoData) {
        InputStream in = videoData.getInputStream
        // ... do something ...
        // save to disk
    }
}

Additionally, we also need configure a Multipart configuration in our Application so that Spring knows that we are expecting multipart data, and to allow our controllers to receive said files. This is also where we would provide information about how big we expect these files to be and other things to decide if the client is uploading too much data or initiating an attack.

public class Application {
  @Bean
  public MultipartConfigElement getMultipartConfig() {
      MultipartConfigFactory f = ...;
      f.setMaxFileSize(2000);
      f.setMaxRequestSize(...);
      return f.createMultipartConfig;
  }

  public static void main(String[] args) { ... }
}

The @Bean annotation will be discussed when we talk about dependency injection. In this specific case, it lets Spring know that the method provides some piece of configuration information that it needs to know about. It will then call the method, and then use the MultipartConfigElement object it receives to configure part of the application.

Generating Responses with the ResponseBody Annotation

Considering our ContactsController, we've discussed how to accept requests in the right format, but we also need to add the @ResponseBody annotation in order to tell Spring to automatically serialize our return type to JSON and add it to the response body. Note that this step can be omitted when the controller is annotated with @RestController as Spring automatically does the conversion for all mapped methods in that case.

@RequestMapping("/search")
@ResponseBody
public Contact search(...) {
    //... find contact ...
    return contact;
}

Custom Data Marshalling with Jackson Annotations

The Jackson Project is the open source JSON library that Spring uses to convert JSON to Java objects and vice versa. So when a request comes in, Spring uses Jackson to convert each parameter into the correct type, and when a response is ready, Jackson takes the return object for the method and then serializes it to JSON for sending to the client.

Let's say we have a custom type called Video that is created for our video web service.

public class Video {
    private String title;
    private String url;
    private long duration;
    private MyCustomType ct;
    private IgnoredField it;

    @JsonIgnore
    public IgnoredField getIt() { return it; }

    // ... Getters/Setters ...
}

And we would like to map JSON into instances of this object.

{
    "title": "Coursera",
    "duration": 1234,
    "url": "https://...",
    "MyCustomType": {...}
}

By default, Jackson assumes that the Video type has a constructor that takes no arguments and uses it to construct a new instance. It will then walk through each key-value pair in the JSON and calls their respective setter methods by comparing keys to the setter names while also converting the values to the appropriate type based on the method signature. By default, Jackson will convert keys to title case ("duration" -> "Duration") and add a "get" or "set" to the front depending on what it needs to do. It also works in the other direction. Fields can also be selectively ignored by Jackson using the @JsonIgnore annotation.

Jackson can also take an entire hierarchy of types and convert them as well. When Jackson sees another custom type like MyCustomType, it will automatically construct an instance of MyCustomType following the same process and then pass it to Video. Jackson will of course throw exceptions when there are unexpected or missing fields in the JSON.

Spring Boot

In order to set up the controllers that we need for our application from scratch, there is some infrastructure that we need to configure beforehand such as setting up the web container using Tomcat or Jetty, configuring the routing in the web container via the web.xml file, instantiate and set up the dispatcher servlet to route requests to the various controllers. All of this is a lot of work, especially for small services. However, there is a sub-project within Spring called Spring Boot that is designed to automate all this for us so that we can focus on writing our controllers and the logic in them.

We can specify as much or as little as we want about the configuration of the web container and other components, and Spring Boot will fill in the blanks and build everything else around it.

Basic Application Structure

The most basic structure of a Spring Boot application includes our controllers and an application file Application.java. The application file can have any arbitrary name as long as it is annotated properly, and it defines the minimum configuration of the application that we are interested in defining.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

Once we run the application with our specifications, Spring Boot takes over and sets up the rest of the infrastructure:

  1. Set up the web container.
  2. Discover our controllers.
  3. Set up the dispatcher servlet.
  4. Any other configurations such as connecting to Databases etc.

Our Video Service Using Spring Boot

In my previous blog about Servlets, we looked at how a basic service for uploading Videos is set up using the traditional servlet and web.xml structure, and how much overhead was required when writing the doGet() or doPost() methods. Here we can see how different our code looks with Spring abstracting away all the boilerplate for us.

@Controller
public class VideoService {
    private List<Video> videos = new ArrayList<>();

    // Equivalent to the doPost method
    @RequestMapping(value="/video", method=RequestMethod.POST)
    public @ResponseBody boolean addVideo(@RequestBody Video v) {
        return videos.add(v);
    }

    // Equivalent to the doGet method
    @RequestMapping(value="/video", method=RequestMethod.GET) 
    public @ResponseBody List<Video> getVideoList() {
        return videos;
    }
}

As you can see, it is far smaller and easier to understand. By letting Spring do the heavy lifting through its automatic data marshalling via the @RequestBody and @ResponseBody annotations, we free up the method to only contain the business logic that we are interested in. This also somewhat decouples our code from the idea of just being a HTTP requests handler and can be used as a plain Java class. Whenever the conversion process of @RequestBody throws an exception, Spring will automatically return a 4xx error to the client, letting them know that they sent an invalid request. Similarly, if our method throws an exception, Spring will automatically return a 5xx error to the client.

Instead of a web.xml file, we now use our Application.java file.

import org.springframework.boot.SpringApplication;

// Tell Spring that this object represents a Configuration for the application
@Configuration
// Tell Spring to turn on WebMVC (eg, set up web container, enable the DispatcherServlet
// so that requests can be routed to our Controllers)
@EnableWebMvc
// Tell Spring to go and scan out controller package (and all sub-packages ) to
// find any Controllers or other components that are part of our application.
// Any class in this package that is annotated with @Controller is going to be
// automatically discovered and connected to the DispatcherServlet.
@ComponentScanning("org.magnum.mobilecloud.video.controller")
// Tell Spring to automaticall inject any dependencies that are marked in
// our classes with @Autowired
@EnableAutoConfiguration
public class Application {
    // Tell Spring to launch our app
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

The four annotations in the file basically sets up all the infrastructure that is needed to run our application. We can also simply replace all these annotations with just @SpringBootApplication.