Pitfall of Go HTTP Header Operation

Use example to explain common mistakes when operating HTTP Header in Go

ta-ching chen

4 minute read

 Table of Contents

Preface 

Most of the recent works are working on manipulating HTTP header in Go and noticed some common mistakes people may fall into while in development. So following we will go through some examples to see how to prevent and resolve these problems.

All sample code can be found at Github Repo。

ResponseWriter.Write causes client receives wrong HTTP status code 

First, let’s see the following function and guess what HTTP status code user will receive.

http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
    w.Write([]byte("foobar"))
    w.WriteHeader(http.StatusBadRequest)
})

400 Bad Request? No, the answer is 200 OK。

$ curl -i http://127.0.0.1:8080
HTTP/1.1 200 OK
Date: Thu, 20 Dec 2018 06:31:33 GMT
Content-Length: 6
Content-Type: text/plain; charset=utf-8

foobar

And server prints following log.

2018/12/20 14:31:33 http: multiple response.WriteHeader calls

The reason caused this problem is the order of invocation of WriteHeader and Write.

The official doc has following comment:

If WriteHeader is not called explicitly, the first call to Write will trigger an implicit WriteHeader(http.StatusOK).

That means if you call Write first, the go server will set status code to http.StatusOK regardless of the real status code set by WriteHeader later.

So next time, if you want to return any status code other than 200 OK, always call WriteHeader first to prevent such problem.

Header.Set and Header.Add are similar but different 

In this part you need server.go and proxy.go to launch a HTTP server and a proxy server. What proxy does here is when a request comes to proxy, the proxy will add some metadata to request and proxy it to backend HTTP server.

http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
    h1 := r.Header["Header1"]
    h2 := r.Header["Header2"]
    msg := fmt.Sprintf("header1: %v, header2: %v", h1, h2)
    w.Write([]byte(msg))
})

func (p Proxy) handle(w http.ResponseWriter, r *http.Request) {
    // We may want to attach metadata to request header
    r.Header.Add("Header1", "bar")
    r.Header.Set("Header2", "foo")
    p.proxy.ServeHTTP(w, r)
}

The difference between Header1 and Header2 is proxy use different functions (Add, Set) to modify the header value.

Again, let’s guess what’s the output of two headers.

$ curl -i http://127.0.0.1:8080 -H 'header1: 123' -H 'header2: 456'
HTTP/1.1 200 OK
Content-Length: 34
Content-Type: text/plain; charset=utf-8
Date: Thu, 20 Dec 2018 06:53:54 GMT

header1: [123 bar], header2: [foo]

The answer is Header1 returns a map contains two elements while Header2 contains only 1.

Here are the explanation from docs for Add and Get functions.

Add adds the key, value pair to the header. It appends to any existing values associated with key.

Set sets the header entries associated with key to the single element value. It replaces any existing values associated with key.

So always make sure to select the right one for your use cases.

Notice the return value of Header.Get 

What value will we get, if using Header.Get to get value of Header1?

r.Header.Get("Header1")

The answer is 123 instead of [123, bar].

func (h Header) Get(key string) string {
    return textproto.MIMEHeader(h).Get(key)
}

func (h MIMEHeader) Get(key string) string {
    if h == nil {
        return ""
    }
    v := h[CanonicalMIMEHeaderKey(key)]
    if len(v) == 0 {
        return ""
    }
    return v[0]
}

From stdlib implementation you can that Header.Get only returns the first element in list. So, if you want to retrieve whole value of a header, you have to access Header directly.

Letter case of header key 

What’s return value of V1 and V2 ?

http.HandleFunc("/", func (w http.ResponseWriter, r *http.Request) {
    v1 := r.Header["foo"]
    v2 := r.Header[textproto.CanonicalMIMEHeaderKey("foo")]

    msg := fmt.Sprintf("V1: %v, V2: %v", v1, v2)

    w.Write([]byte(msg))
})

Ha, this time only V2 has value returned.

$ curl -i http://127.0.0.1:8080 -H 'foo: 123'
HTTP/1.1 200 OK
Date: Thu, 20 Dec 2018 07:14:07 GMT
Content-Length: 17
Content-Type: text/plain; charset=utf-8
  
V1: [], V2: [123]

The reason is that any request header key goes into go http server will be converted into case-sensitive keys.

The canonicalization converts the first letter and any letter following a hyphen to upper case; the rest are converted to lowercase. For example, the canonical key for “accept-encoding” is “Accept-Encoding”.

So you can call textproto.CanonicalMIMEHeaderKey to do the key conversion before accessing Header map.

For most of cases, Header.Get is a better option since it does the conversion implicitly unless you want to access map directly.

Reference 

See Also

To reproduce, republish or re-use the content, please attach with link: https://tachingchen.com/
comments powered by Disqus