How to find out the HTTP header length of a packet?

Tag: http , wireshark , netmon Author: ouyangbeyond Date: 2011-04-02

I know how to do it manually (by looking at the hex dump). How can I obtain the same automatically? Do I have to use the APIs? I have both wireshark and Microsoft network monitor.

What do you mean 'automatically', from an application? If you just mean figuring out what part of the capture is the HTTP header, etc., Wireshark should automatically dissect the packets. Find any HTTP data packet, right-click and select "Follow TCP Stream" and it will show the HTTP traffic with the headers clearly readable.
@Tim: I want to know the HTTP Header Length in bytes. I can see the HTTP conversation, but how do I put the HTTP header length as a column lets say?
I don't know what you mean "as a column". Basically you need to search the TCP stream from the beginning of the HTTP request to the first double-newline (\n\n or \r\n\r\n). The index where that's found is the length of the header. If you want this to show up within Wireshark, you'll need to develop a plug-in or something. If you're dumping the data to excel tables or something, then you'll need to compute the length as I described.
You can choose the columns to display:Edit->Preference->UserInterface (Columns). How can I develop a plugin for displaying this in Wireshark? Can you point to some tutorials or useful links.

Best Answer

This can be achieved simply with a Lua dissector that adds an HTTP header field to the packet tree, allowing you to filter for it, as shown in this screenshot:

enter image description here

Copy this Lua script into your plugins directory (e.g., ${WIRESHARK_HOME}/plugins/1.4.6/http_extra.lua), and restart Wireshark (if already running).

do
        local http_wrapper_proto = Proto("http_extra", "Extra analysis of the HTTP protocol");
        http_wrapper_proto.fields.hdr_len = ProtoField.uint32("http.hdr_len", "Header length (bytes)")

        -- HTTP frames that contain a header usually include the HTTP
        -- request method or HTTP response code, so declare those here
        -- so we can check for them later in the dissector.
        local f_req_meth    = Field.new("http.request.method")
        local f_resp_code   = Field.new("http.response.code")

        local original_http_dissector
        function http_wrapper_proto.dissector(tvbuffer, pinfo, treeitem)
                -- We've replaced the original http dissector in the dissector table,
                -- but we still want the original to run, especially because we need 
                -- to read its data. Let's wrap the call in a pcall in order to catch
                -- any unhandled exceptions. We'll ignore those errors.
                pcall(
                    function()
                        original_http_dissector:call(tvbuffer, pinfo, treeitem)
                    end
                )

                -- if the request method or response code is present,
                -- the header must be in this frame
                if f_req_meth() or f_resp_code() then

                        -- find the position of the header terminator (two new lines),
                        -- which indicates the length of the HTTP header, and then add
                        -- the field to the tree (allowing us to filter for it)
                        local hdr_str = tvbuffer():string()
                        local hdr_len = string.find(hdr_str, "\r\n\r\n") or string.find(hdr_str, "\n\n\n\n")
                        if hdr_len ~= nil then
                            treeitem:add(http_wrapper_proto.fields.hdr_len, hdr_len):set_generated()
                        end
                end
        end

        local tcp_dissector_table = DissectorTable.get("tcp.port")
        original_http_dissector = tcp_dissector_table:get_dissector(80) -- save the original dissector so we can still get to it
        tcp_dissector_table:add(80, http_wrapper_proto)                 -- and take its place in the dissector table
end

comments:

Thanks a lot for replying. I followed your steps but I don't see http header length in the packet description. I am working on Windows.
@Bruce: 1. What version of Wireshark are you running? 2. When you go to menu "Tools", do you see a sub-menu "Lua"? If not, then Lua is disabled (enabled by default in the standard installation). 3. The HTTP header length field is only added to the packet description if the packet contains the HTTP header. Did you verify that?
@Bruce: btw, I tested this script on Wireshark 1.4.6 on Windows XP SP3.
I have Wireshark 1.2.7. I don't see the Lua sub-menu. I will reinstall with Lua enabled.
Cool it works now!

Other Answer1

Unfortunately, although you can create custom columns, the data you want in that column is not currently generated by the HTTP protocol decoder. Of course, there may be other tools that I'm not familiar with which can do this today, but as far as Wireshark is concerned you would have to add that functionality.

There are some good resources on creating Wireshark plugins, e.g.:

http://simeonpilgrim.com/blog/2008/04/29/how-to-build-a-wireshark-plug-in/

http://www.wireshark.org/docs/wsdg_html_chunked/ChDissectAdd.html

http://www.codeproject.com/KB/IP/custom_dissector.aspx

And here's a video describing how to add a field that's exposed by a protocol decoder as a custom column:

http://www.youtube.com/watch?v=XpUNXDkfkQg

The thing is, you don't want to re-implement the HTTP protocol decoder.

What I would do is find the source code for the built-in HTTP decoder and look at adding a new field such as http.header_length just like the existing http.content_length:

img

I haven't looked at the code, but I would guess that this is a pretty easy thing to add. If you submit a patch to the Wireshark team they will probably also include your new field in the next release.

Other Answer2

I've found that this way of calling previous dissector in chain somehow interferre with HTTP packet reassembly done for 'chunked' transfer encoding. That is if your response has 'Transfer-Encoding: chunked' header, the original HTTP dissector tries to reassemble the data and if you hook it over with such http_wrapper, then reassembling fails.

For example, this makes http statistics fail too. Statistics/HTTP/Packet Counter would give you, say 6 requests and 4 responses, which is not the case =)

One should better install such kind of 'added value' dissectors with 'register_postdissector' API call or test for reassembling logic carefully.