In this chapter, we’re going to reuse some of what we learned in the previous chapter to make our mapping of the web application more relevant!
Map the Website
You’ll recall that, in the chapter Collect More Information with Active Reconnaissance, we looked for other applications or systems on our target that were associated with our initial target application. We learned that these could be out of scope, so we would need to check with our client whether we were authorized to look at them. If we were, we also needed to update the scope document accordingly.
We’re going to adopt the same approach in this chapter, but specifically for the target application. This is definitely not out of scope, and in fact, it’s essential if we want to be sure we haven’t missed anything!
Let’s get started!
We already explored the application a little in the discovery phase, to familiarize ourselves with it. Now we’re going to start digging a little deeper and taking note of any interesting behaviors.
It’s time to activate the interception proxy and explore the application. This is the perfect opportunity to take notes on interesting modules, error pages containing technical information, and so on.
Speed Up Your Mapping
Mapping the application involves two separate activities:
Mapping the known parts by jumping from link to link (known as crawling or spidering).
Searching for parts that are more “hidden” with no links pointing to them. This is known as enumeration.
It’s best to do spidering manually to map the known parts, as this helps us understand the entire structure of the application. To give you a more complete picture, you could try a tool like gospider or use Burp Suite’s built-in scanner if you have the Professional version.
When it comes to enumerating hidden parts, you can also do the work manually by:
editing the URL in the search bar.
adding, one by one, the various options you think of or those in the list you’ve downloaded.
Another option is to automate this task by:
creating a small bash or Python script to run through a series of cURL requests.
using one of the many tools available on the web: Burp Intruder (Professional version required), ffuf, dirbuster, the list goes on!
Personally, I prefer to use ffuf, mostly just because I’ve learned how to use it, and it does what I need. The screenshots and video tutorials in this chapter therefore feature this tool.
Let’s take a look at how you can use ffuf with a wordlist to discover web content:
Over to You!
Challenge
Can you find the page requiring authentication?
You need to find this page using ffuf to successfully complete this challenge.
You don’t have to find the flag—that’s outside the scope of the exercise—but if you’d like to finish the challenge, feel free!
Solution
Since this exercise is based on a Root Me challenge, we can’t give you a cookie-cutter solution, because the philosophy behind hacking is that you have to tinker around to learn. That said, this chapter includes everything you need to complete this exercise successfully. Read through it again, if you need to, and watch the demonstration video.
By the way, the video above includes an important hint to help you: We need to include certain options so that Root Me doesn’t block our tools! For ffuf, these options are: -t 7 -rate 70 -H "User-Agent: Firefox"
Avoid the Pitfalls
Remember, tools are not a magic bullet!
Make Sure Your Automation Works Properly
As with other types of scanning, there are always pitfalls and false positives—or worse, false negatives! These are automation tools, after all.
For example, if you pass them a request with a particular header missing, the enumeration will always return the same HTTP code (e.g., 404 or 403 ). This will happen if you incorrectly copy the headers that are mandatory for the application.
When I’m not sure how things work, I pass all requests from these tools through Burp. Performance may suffer, but at least I’m sure of what I’m sending to the application!
Analyze the Application Carefully Before Automating
When you attack a REST API, for example, the specification requires it to respond with anHTTP 403code if you don’t have permission to access the target page. However, some APIs will respond with anHTTP 200code and will only give the “real” code—HTTP 403 —in the body of the response! So, we won’t have access to this page.
Watch Out for Protection Mechanisms
You should also keep in mind that the website may be protected by devices such as a Web Application Firewall (WAF). A WAF can detect abnormal behavior and ban a particular client, for example.
Let’s Recap!
You can find hidden API pages or endpoints by:
mapping the target application with a crawling or spidering tool.
supplementing this mapping with an enumeration from a predefined list.
This helps uncover features that are not always publicly exposed.
Clients sometimes forget about these features. They may therefore be less protected and prime targets for us!
To find them, we need a tool to automate the enumeration and a good list that’s appropriate for the target (e.g., a PHP list for a PHP application, not a Java list).
But make sure you configure the tool properly and don’t blindly trust it!
That’s the end of this part of the course. The next part is going to be intense—but exciting—as we’ll finally tackle how to find vulnerabilities in the application!
