Your tools

In the last lesson you sent every part of a request by hand. This one is about the three tools you will use to do that for real, every day: your browser's developer tools, curl, and an intercepting proxy. They all show you the same thing, an HTTP request and the response it got, and each is better at a different job. Knowing which to reach for is most of what "being fast" means.

The browser's developer tools#

Press F12 (or right-click → Inspect) and you get a panel that has been watching your browser the whole time. Two tabs matter for us.

Network. Every request the page makes, one row each. Reload with it open and you see the page load itself: the HTML, then every stylesheet, script and image, then whatever the JavaScript asks for afterwards. Click a row and you get four things worth knowing:

• Headers: the request line, the request headers you sent, and the response status and headers you got back.
• Payload: the body you sent, if any (a form post, some JSON).
• Response: the raw body the server returned, before the page did anything with it.
• Timing: how long each part took.

Two settings make it usable. Preserve log stops the list clearing on every navigation, so a login redirect does not wipe the request you wanted to read. The filter row lets you show only Fetch/XHR, which hides the images and fonts and leaves the requests the application actually makes.

The single most useful thing in this tab is hidden in the right-click menu on any request: Copy → Copy as cURL and Copy as fetch. It hands you the exact request, every header and cookie included, as a command you can paste into a terminal or the Console and change. You will use this constantly: see a request in the browser, copy it as curl, and now you can replay it a hundred times with one field changed.

Console. A JavaScript prompt running in the page. Because it runs as the page, fetch() from here sends the page's cookies automatically, so it is the quickest way to replay a request as the logged-in user:

fetch('/api/orders/1001').then(r => r.json()).then(console.log)

Application. Cookies, local storage, session storage: everything the site has stored in your browser. This is where you read the cookie that is keeping you logged in, and where you can change it.

The limit of DevTools is important, because it is the reason the other two tools exist. DevTools lets you watch, and replay, but not intercept. It cannot pause a request the page is about to send and let you edit it before it leaves. And there are some request headers the browser will simply not let page JavaScript set at all.

curl#

curl is a command-line program that speaks HTTP and does exactly what you tell it, with no page, no JavaScript, and nothing added behind your back. That is its whole value: there is nothing between you and the bytes.

The flags you will use most:

curl -i https://example.com/                 # -i: print the response headers, not just the body
curl -X POST https://example.com/thing       # -X: set the method
curl -H "X-Custom: 1" https://example.com/   # -H: add a request header (repeatable)
curl -d "a=1&b=2" https://example.com/login  # -d: send a body (and imply POST)
curl -H "Content-Type: application/json" -d '{"a":1}' https://example.com/api
curl --cookie "session=abc" https://example.com/account   # send a cookie
curl -L https://example.com/                 # -L: follow redirects
curl -v https://example.com/                 # -v: show the whole conversation, request included

curl reaches for the terminal, so it is the tool for anything you want to repeat, script, or send in a way a browser would not: a header the browser forbids, a malformed body, a method no form would ever use. When you want to know exactly what went on the wire, with nothing helping you, this is it.

The one thing to remember: curl sends nothing you did not ask for. No cookies unless you pass them, no User-Agent beyond its own, no headers a browser adds automatically. That is a feature, but it means a request copied from the browser sometimes needs its cookies added back before it works. Copy as cURL does that for you.

An intercepting proxy#

The third tool sits between your browser and the server. Your browser is told to send everything to the proxy; the proxy shows you each request, lets you stop it, edit any part, and forward it, then does the same with the response. The main ones are Burp Suite (the industry default, free Community edition), Caido (newer, lighter), and mitmproxy (terminal-based).

Three parts of it you will live in:

• Intercept: pause each request in flight and edit it before it goes. This is the thing DevTools cannot do.
• HTTP history: every request that passed through, searchable, long after it happened.
• Repeater: send one request over and over, editing it each time. This is where most manual testing actually happens: you find an interesting request, send it to Repeater, and change one thing at a time.

Why it needs to install a certificate. In the first lesson, HTTPS was HTTP inside a TLS tunnel that the browser checks with a certificate. A proxy that wants to read your HTTPS traffic has to sit inside that tunnel, which means being trusted by your browser. So it generates its own certificate authority and you install it, once, telling your browser "trust certificates this proxy signs." Now the proxy can read and change HTTPS traffic, because to your browser it is a trusted server. This is exactly the "man in the middle" that a certificate is supposed to prevent; you are allowing it, on your own machine, on purpose.

The proxy is the daily driver for real testing because it combines the browser's convenience with curl's control, and because once a page's JavaScript is making requests you did not write, the proxy is the only one of the three that lets you change them as they happen.

The headers a browser will not let a page set#

Here is the concrete reason a tester ever leaves the browser. When page JavaScript makes a request with fetch(), the browser refuses to let it set certain headers, no matter what the code says. These are the "forbidden headers", and they include:

• Host
• Origin
• Referer
• Cookie (the page can set cookies a different way, but not this header directly)
• User-Agent
• Content-Length, and a handful of others

The browser controls these because a page lying about them would break the security assumptions of other sites. But a security tester often needs to send a specific one of these, to see how the server behaves. A page cannot. curl and a proxy have no such rule: to them, every header is just text you typed.

That is the dividing line. If you can do it in the browser, do, it is faster. The moment you need to send something the browser will not, or change a request after the page built it, you move to curl or the proxy. The lab makes you cross that line once, so you feel where it is.

Which one, when#

You do not need all three set up today. DevTools you already have, and curl is one install. The proxy is the next lesson's job, because setting it up (and installing that certificate) is a small project of its own. For the lab below, DevTools and curl are enough.

Now do the lab#

The lab is a tiny site called Lookout. It has three steps, and each one is best, or only, done with a different tool: one you find by watching the Network tab, one you can only send from curl because it needs a header the browser forbids, and one you carry back in a header of your own. When you finish it you will have used two of the three tools on one target, and felt exactly why the second one exists.