Pocket Gophers

Limit HTTPS Listener

go1.6.3

From ”how to apply netutil.LimitListener() to ListenAndServeTLS()'s listener? how to fetch ListenAndServeTLS()'s listener?” on golang-nuts:

I found an example to limit the number of connections for a listener:

import "golang.org/x/net/netutil"
//Within net/netutil/listen.go
//func LimitListener(l net.Listener, n int) net.Listener
connectionCount := 2
l, err := net.Listen("tcp", ":8000")
if err != nil {
	log.Fatalf("Listen: %v", err)
}
defer l.Close()
l = netutil.LimitListener(l, connectionCount)
log.Fatal(http.Serve(l, nil)

BUT I’m not using net.Listen() and http.Serve(). I’m using the ListenAndServeTLS().

A closer look into ListenAndServeTLS:

func (srv *Server) ListenAndServeTLS(certFile, keyFile string) error
{
...
tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
return srv.Serve(tlsListener)
}

How can I fetch the srv’s tlsListener and then apply the netutil.LimitListener()? ie. netutil.LimitListener(srv.tlsListener, connectionCount)?

tlsListener is not stored in http.Server, and so cannot be fetched. However, it is still possible by following the pattern shown by ListenAndServeTLS:

  1. Create an http.Server
  2. Create an net.Listener
  3. Serve

This pattern can also be seen from Server.Serve:

func (srv *Server) Serve(l net.Listener) error

With this realization, the difficulty is creating a net.Listener like ListenAndServeTLS does. As this is more tricky than I would like, I wrote down my process.

Getting and Running the Example Code

You can follow along by getting and running the example code.

Use go get to download the example code:

go get -d pocketgophers.com/limit-https-listener

The code will be downloaded to:

$GOPATH/pocketgophers.com/limit-https-listener

Each example is in a separate file and can be ran like:

go run example.go

Replace example.go with the example you want to run.

A Working Example

The first step is to create a working example. From the problem description I assume that

func (srv *Server) ListenAndServeTLS(certFile, keyFile string) error

is used. This means I need to create a *Server, a certFile, and a keyFile.

The files referenced by certFile and keyFile can be created by running:

go run `go env GOROOT`/src/crypto/tls/generate_cert.go --host localhost

This creates cert.pem and key.pem, a self-signed certificate. I don’t include these files with the example code because the certificate will expire at some point and they are easy to generate. Your browser should ask you to accept this certificate the first time you load it.

Now that I have the certificate, the full example is:

example.go
package main

import (
	"fmt"
	"log"
	"net/http"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	err := srv.ListenAndServeTLS("cert.pem", "key.pem")
	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

A few notes on this implementation:

Expanding ListenAndServeTLS

I try to be mechanical when expanding functions. Its usually starts out a bit messy, so I clean it up after it is working. The expansion process usually follows this process:

  1. Create variables for the arguments
  2. Paste in the functions code
  3. Deal with return values and flow so that the flow is the same as if the function was called and then returned
  4. Deal with name conflicts

This starts out easily enough. I create variable for the arguments, paste in the function code, replace return err with log.Fatalln(err), and replace the final return to err = .

expanded.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	certFile := "cert.pem"
	keyFile := "key.pem"

	addr := srv.Addr
	if addr == "" {
		addr = ":https"
	}

	// Setup HTTP/2 before srv.Serve, to initialize srv.TLSConfig
	// before we clone it and create the TLS Listener.
	if err := srv.setupHTTP2(); err != nil {
		log.Fatalln(err)
	}

	config := cloneTLSConfig(srv.TLSConfig)
	if !strSliceContains(config.NextProtos, "http/1.1") {
		config.NextProtos = append(config.NextProtos, "http/1.1")
	}

	configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
	if !configHasCert || certFile != "" || keyFile != "" {
		var err error
		config.Certificates = make([]tls.Certificate, 1)
		config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
		if err != nil {
			log.Fatalln(err)
		}
	}

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
	err = srv.Serve(tlsListener)

	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

However, when I go run expanded.go I get:

# command-line-arguments
./expanded.go:31: srv.setupHTTP2 undefined (cannot refer to unexported field or method http.(*Server)."".setupHTTP2)
./expanded.go:35: undefined: cloneTLSConfig
./expanded.go:36: undefined: strSliceContains
./expanded.go:55: undefined: tcpKeepAliveListener

Each of these problems need to be solved.

srv.setupHTTP2

srv.setupHTTP2 is defined as:

func (srv *Server) setupHTTP2() error {
	srv.nextProtoOnce.Do(srv.onceSetNextProtoDefaults)
	return srv.nextProtoErr
}

This is annoying because it refers to another exported item: nextProtoOnce. From Server’s definition:

nextProtoOnce 	sync.Once // guards initialization of TLSNextProto in Serve

The use of sync.Once implies setupHTTP2 is expected to be called more than once, so I search for it and notice Serve calls srv.setupHTTP2. Since sync.Once ensures the effect of calling setupHTTP2 only happens once, there is no reason for it to be called twice: once by me and then by Serve. Therefore the code can be deleted.

setupHTTP2.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	certFile := "cert.pem"
	keyFile := "key.pem"

	addr := srv.Addr
	if addr == "" {
		addr = ":https"
	}

	config := cloneTLSConfig(srv.TLSConfig)
	if !strSliceContains(config.NextProtos, "http/1.1") {
		config.NextProtos = append(config.NextProtos, "http/1.1")
	}

	configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
	if !configHasCert || certFile != "" || keyFile != "" {
		var err error
		config.Certificates = make([]tls.Certificate, 1)
		config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
		if err != nil {
			log.Fatalln(err)
		}
	}

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
	err = srv.Serve(tlsListener)

	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

go run setupHTTP2.go leaves me with the following errors:

# command-line-arguments
./setupHTTP2.go:29: undefined: cloneTLSConfig
./setupHTTP2.go:30: undefined: strSliceContains
./setupHTTP2.go:49: undefined: tcpKeepAliveListener

cloneTLSConfig

From is name, I guess that cloneTLSConfig does what it says, but check it anyway and find:

cloneTLSConfig returns a shallow clone of the exported fields of cfg, ignoring the unexported sync.Once, which contains a mutex and must not be copied.

My Server does not set TLSConfig, so I look to see what is done when it is nil and find:

if cfg == nil {
	return &tls.Config{}
}

Therefore I can replace the call to cloneTLSConfig with &tls.Config{}

cloneTLSConfig.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	certFile := "cert.pem"
	keyFile := "key.pem"

	addr := srv.Addr
	if addr == "" {
		addr = ":https"
	}

	config := &tls.Config{}
	if !strSliceContains(config.NextProtos, "http/1.1") {
		config.NextProtos = append(config.NextProtos, "http/1.1")
	}

	configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
	if !configHasCert || certFile != "" || keyFile != "" {
		var err error
		config.Certificates = make([]tls.Certificate, 1)
		config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
		if err != nil {
			log.Fatalln(err)
		}
	}

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
	err = srv.Serve(tlsListener)

	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

go run cloneTLSConfig.go results in:

# command-line-arguments
./cloneTLSConfig.go:30: undefined: strSliceContains
./cloneTLSConfig.go:49: undefined: tcpKeepAliveListener

strSliceContains

From its name, I guess that strSliceContains checks to see if a string slice contains a certain string. The implementation confirms this:

func strSliceContains(ss []string, s string) bool {
	for _, v := range ss {
		if v == s {
			return true
		}
	}
	return false
}

But, I ask myself, does my config.NextProtos contain http/1.1? It can’t because config.NextProtos is not set, and therefore nil. Therefore the interior of the if statement would always execute. So I remove the if statement, leaving the interior. I notice config.NextProtos is set with append, but I known that append will not work because config.NextProtos is nil, so I replace it with a literal.

strSliceContains.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	certFile := "cert.pem"
	keyFile := "key.pem"

	addr := srv.Addr
	if addr == "" {
		addr = ":https"
	}

	config := &tls.Config{}
	config.NextProtos = []string{"http/1.1"}

	configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
	if !configHasCert || certFile != "" || keyFile != "" {
		var err error
		config.Certificates = make([]tls.Certificate, 1)
		config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
		if err != nil {
			log.Fatalln(err)
		}
	}

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
	err = srv.Serve(tlsListener)

	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

go run strSliceContains.go leave me with:

# command-line-arguments
./strSliceContains.go:47: undefined: tcpKeepAliveListener

tcpKeepAliveListener

tcpKeepAliveListener is defined as:

// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
// connections. It's used by ListenAndServe and ListenAndServeTLS so
// dead TCP connections (e.g. closing laptop mid-download) eventually
// go away.
type tcpKeepAliveListener struct {
	*net.TCPListener
}

func (ln tcpKeepAliveListener) Accept() (c net.Conn, err error) {
	tc, err := ln.AcceptTCP()
	if err != nil {
		return
	}
	tc.SetKeepAlive(true)
	tc.SetKeepAlivePeriod(3 * time.Minute)
	return tc, nil
}

Unlike the previous problems, this one cannot be solved by deleting the reference to tcpKeepAliveListener while maintaining the same functionality. Instead, I copy it into my source.

tlsKeepAliveListener.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
	"time"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	certFile := "cert.pem"
	keyFile := "key.pem"

	addr := srv.Addr
	if addr == "" {
		addr = ":https"
	}

	config := &tls.Config{}
	config.NextProtos = []string{"http/1.1"}

	configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
	if !configHasCert || certFile != "" || keyFile != "" {
		var err error
		config.Certificates = make([]tls.Certificate, 1)
		config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
		if err != nil {
			log.Fatalln(err)
		}
	}

	ln, err := net.Listen("tcp", addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)
	err = srv.Serve(tlsListener)

	if err != nil {
		log.Fatalln(err)
	}
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
// connections. It's used by ListenAndServe and ListenAndServeTLS so
// dead TCP connections (e.g. closing laptop mid-download) eventually
// go away.
type tcpKeepAliveListener struct {
	*net.TCPListener
}

func (ln tcpKeepAliveListener) Accept() (c net.Conn, err error) {
	tc, err := ln.AcceptTCP()
	if err != nil {
		return
	}
	tc.SetKeepAlive(true)
	tc.SetKeepAlivePeriod(3 * time.Minute)
	return tc, nil
}

After all that work, it finally compiles and runs.

Cleanup

The last step in expanding ListenAndServeTLS is to cleanup the code.

Starting with:

addr := srv.Addr
if addr == "" {
	addr = ":https"
}

Since srv.Addr is localhost:8000 the interior of the if never executes, so I delete the code and replace the remaining reference to addr with srv.Addr.

Next is:

config := &tls.Config{}
config.NextProtos = []string{"http/1.1"}

I prefer to setup as many of the fields of a struct when I create it as I can. Therefore I move the setting of NextProtos into the literal:

config := &tls.Config{
	NextProtos: []string{"http/1.1"},
}

Next up is the certificate configuration:

configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil
if !configHasCert || certFile != "" || keyFile != "" {
	var err error
	config.Certificates = make([]tls.Certificate, 1)
	config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)
	if err != nil {
		log.Fatalln(err)
	}
}

Since the interior of the if will always run, I remove the conditional and the setting of configHasCert, which has no other uses. I also replace certFile and keyFile with their values and delete those variables since this is the only place they are used. This leaves me with:

var err error
config.Certificates = make([]tls.Certificate, 1)
config.Certificates[0], err = tls.LoadX509KeyPair("cert.pem", "key.pem")
if err != nil {
	log.Fatalln(err)
}

I don’t move the initial setting of config.Certificates into the Config literal because its creation and setup in the next line are closely related. Splitting the setup would require me to look in two places should I need to change it.

Last of all, I reduce

err = srv.Serve(tlsListener)

if err != nil {
	log.Fatalln(err)
}

to my preferred idiom:

log.Fatalln(srv.Serve(tlsListener))

With the cleanup completed, I have:

cleaned.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
	"time"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	config := &tls.Config{
		NextProtos: []string{"http/1.1"},
	}

	var err error
	config.Certificates = make([]tls.Certificate, 1)
	config.Certificates[0], err = tls.LoadX509KeyPair("cert.pem", "key.pem")
	if err != nil {
		log.Fatalln(err)
	}

	ln, err := net.Listen("tcp", srv.Addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)

	log.Fatalln(srv.Serve(tlsListener))
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
// connections. It's used by ListenAndServe and ListenAndServeTLS so
// dead TCP connections (e.g. closing laptop mid-download) eventually
// go away.
type tcpKeepAliveListener struct {
	*net.TCPListener
}

func (ln tcpKeepAliveListener) Accept() (c net.Conn, err error) {
	tc, err := ln.AcceptTCP()
	if err != nil {
		return
	}
	tc.SetKeepAlive(true)
	tc.SetKeepAlivePeriod(3 * time.Minute)
	return tc, nil
}

Adding LimitListener

Now that ListenAndRunTLS is expanded, giving me access to tlsListener, I can finally use LimitListener.

limited.go
package main

import (
	"crypto/tls"
	"fmt"
	"log"
	"net"
	"net/http"
	"time"

	"golang.org/x/net/netutil"
)

func main() {
	log.SetFlags(log.Lshortfile)
	log.Println("started")

	http.HandleFunc("/", hello)

	srv := &http.Server{
		Addr: "localhost:8000",
	}

	config := &tls.Config{
		NextProtos: []string{"http/1.1"},
	}

	var err error
	config.Certificates = make([]tls.Certificate, 1)
	config.Certificates[0], err = tls.LoadX509KeyPair("cert.pem", "key.pem")
	if err != nil {
		log.Fatalln(err)
	}

	ln, err := net.Listen("tcp", srv.Addr)
	if err != nil {
		log.Fatalln(err)
	}

	tlsListener := tls.NewListener(tcpKeepAliveListener{ln.(*net.TCPListener)}, config)

	connectionCount := 2
	limitedListener := netutil.LimitListener(tlsListener, connectionCount)

	log.Fatalln(srv.Serve(limitedListener))
}

func hello(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintln(w, "Hello, 世界")
}

// tcpKeepAliveListener sets TCP keep-alive timeouts on accepted
// connections. It's used by ListenAndServe and ListenAndServeTLS so
// dead TCP connections (e.g. closing laptop mid-download) eventually
// go away.
type tcpKeepAliveListener struct {
	*net.TCPListener
}

func (ln tcpKeepAliveListener) Accept() (c net.Conn, err error) {
	tc, err := ln.AcceptTCP()
	if err != nil {
		return
	}
	tc.SetKeepAlive(true)
	tc.SetKeepAlivePeriod(3 * time.Minute)
	return tc, nil
}

The prototype is complete. I managed to create a tlsListener like the one created in ListenAndServeTLS and then apply netutil.LimitListener to it.

Dig into the Fundamentals of Go

Subscribe to receive a weekly email covering a Go fundamental. Be it the language, its tooling, or its packages, you will learn what you need to know.