My Ideal Service

Posted

This is a set of features that I like to see from services, whether they are personal services to run on a home server or production services powering a business. The common theme of this list is providing enough hooks and documentation so that a lot of standard requirements can be handled by a layer in front of the service. This ensures that these key requirements are handled consistency by trusted tools and removes the requirement to re-implement them in every service.

Use HTTP

Where possible use HTTP. There are times for custom protocols, but HTTP is widely understood, has great tooling, is accessible from the web and is quite performant (with HTTP 2 and 3). Unless there is a really strong justification for a custom protocol it is best to use HTTP. A lot of items later on this list assume HTTP, however if a custom protocol is used these features are still wanted, they will just need to be implemented in the service itself as they can’t be implemented in the reverse-proxy.

Listen on UNIX Sockets

UNIX sockets support filesystem permissions, allowing me to restrict which users and groups can access the service. IP sockets can be accessed by all services on the system. Even on a single-user system where all services are trusted this goes against the principle of least privilege and raises the impact if any service is compromised. This is especially important when doing authentication in a reverse-proxy as local services can bypass it by accessing localhost directly.

I would even argue that most services shouldn’t except to be exposed directly to the internet anyways, it is better to expect a local reverse-proxy to handle rate limiting, access control and logging as well as providing a full-featured battle-hardened and performant HTTP implementation before sending clean HTTP to the service.

NGINX example:

location / {
	proxy_pass http://unix:/run/ipfs-api.sock:;
}

Socket Passing

Services should support accepting their socket from the caller instead of binding the socket themselves. This provides a number of advantages.

If the service only needs one socket (which is the most common case) the simpler inetd protocol can be used. If multiple sockets are required the systemd protocol can be used to support any number of named sockets.

Example of socket-passing via systemd:

[Socket]
ListenStream=/run/ipfs-api.sock
# Only users in the ipfs group may access the IPFS API.
SocketGroup=ipfs
SocketMode=660

Header-based Auth

It is unreasonable for every service to implement every authentication method that a user might want, that would be an NxM problem. Instead it should support relying on a reverse-proxy to provide the authentication information.

The simplest option is passing the username in a request header. This can be set by the reverse-proxy once it has authenticated the user or cleared if the user is not authenticated.

It is also convenient if users are automatically created when a new username is received, however assigning default permissions can be difficult so not all services can offer this without significant effort.

NGINX example:

auth_basic "Restricted Access";
auth_basic_user_file "users.htaccess";
proxy_set_header X-User $remote_user;

For services that support public access a login URL should be clearly documented such that that URL can be intercepted by the reverse-proxy and the auth provided via header. For services that are entirely private (either by design or by administrator choice) the reverse-proxy can be be relied upon to intercept all URLs.

For API-based access it is difficult for the reverse-proxy to intercept the request and present login info. In that case the best approach is allowing users to generate auth-tokens. These auth tokens then should be passed in a uniform way across your API. Good options are a Authorization header containing either a Bearer token or a Basic username and token. Personally I prefer Basic as it allows proxies to determine the username for rate-limiting or blocking users if they are removed. Another option is passing the token in a URL query parameter but this is not recomended because URLs are generally logged and auth tokens shouldn’t be! However no matter how the tokens are passed they should be validated by the service and the request must be rejected if the auth token is invalid or the provided username doesn’t match the token. This should happen even for URLs where the auth token isn’t needed! Otherwise the reverse-proxy can’t assume that requests with a token are authenticated.

Proper Cache Headers

Many services don’t set reasonable cache headers by default. Some services have an “operating guide” describing how to configure a reverse-proxy to apply the correct headers, but why make every user do this when you can just do it once in the service itself? Not only is this configuration service-specific but the service will always have more accurate information about what can be cached and for how long, not only can this be used by clients but it can also be used to apply caching in the reverse-proxy if desired. Services should also generate and check ETags and Last-Modified headers when it makes sense.

Services should not perform their own caching (or should provide a way to turn it off) and rely on a reverse-proxy. This makes it easier to manage the cache across services if desired.

One complication here is controlling local caching vs browser caching. In some cases resources are cacheable, but cheap enough to generate that it is not worth caching on the same machine. NGINX uses the X-Accel-Expires header to override its caching but not all reverse-proxies have a similar option. The best option here is to simply provide a list of URLs that administrators should exclude from local caching.

Structured Logging

Structured logging is great because it is easy to monitor and alert on a service. Good structured logging can even replace the need to emit metrics for a large number of use cases. Some tips for structured logging are below.