So you have been tasked with performing a penetration test of the internet-facing systems of a customer. The security assessment is due soon, and you have to find a reasonable solution to gain information about the customer’s networks and systems as well as targets to attack — all of this in the nick of time.
The first step is always gathering as much useful info as you can, but this process can take up some time. Thankfully there are plenty of solutions if you want to avoid using outdated command-line methods.
For many years now, specialists have turned to OSINT based tools and services, which have proved extremely quick and reliable in penetration testing. In this article, we will give you an overview of what is OSINT and list some of the tools commonly used in penetration testing.
What is Open Source Intelligence?
The term open-source intelligence, or OSINT, was coined in the late 1980s by the US military. They argued that reform of intelligence was needed to cope with the dynamic nature of information, especially at a tactical level on battlefields. The concept of OSINT has traversed into different fields since then and is now commonly used in cybersecurity.
Open-source intelligence is defined as information collected from sources open to the public, mostly through the internet. The term itself does not mean internet information, as info from a public library book can also be viewed as OSINT (a library is a publically available source).
We will look over some of the OSINT tools security experts use daily.
Spyse
Spyse is a recent development in the field of cybersecurity. This search engine scans the internet every couple of days to collect data using OSINT technology, mixed with custom-made algorithms. They store this data in the Spyse database and make it available instantly for users. This alleviates the need to use command-line methods for information gathering, which can be time-consuming.
Spyse provides information about:
-IPv4 (Open ports, banners, protocols, ISP, etc…)
-DNS records
-Domains and subdomains (the biggest subdomain database on the web)
-Digital certificates information
-Autonomous Systems (Number, IPv4/ IPv6 ranges, WHOIS data…)
Google Dorks
Google Dorks have been around for a minute, with specialists using it as far back as 2002. This query-passed, open-source intelligence tool helps users effectively target index or search results.
The flexibility of Dorks makes it one of the most used tools in the field, and the process even has its own nickname — Google Hacking. It uses operators which make the search for information much faster. Here are some operators and indexing options provided by the service:
-Filetype: an operation primarily used to find file types or search for a specific string;
-Intext: an indexing option used for finding text on a particular page;
-Ext: used for searching for a specific extension in a file;
-Inurl: used to find a specific string or word in a URL;
-Intitle: Search for a title for words mentioned in the URL;
The Harvester
Specialists use the Harvester for gathering e-mail accounts, as well as names of subdomains, virtual hosts, open ports and banners, and employee names. All this information is collected from public sources like search engines and PGP key servers.
Recon-Ng
Another tool popular amongst pentesters is Recon-ng. This is another neat reconnaissance tool with a similar interface to Metasploit. You can run Recon-ng from the command line, which places you into a shell-like environment. Here you can edit options, perform reconnaissance, and output results to various report types. They have an interactive console that is loaded with neat features like command completion and contextual help.
SpiderFoot
For Linux and Windows users, we recommend SpiderFoot. This is another high-configuration open-source reconnaissance tool developed with Python. Easily integrable, interactive GUI and a powerful command-line interface makes SpiderFoot a go-to tool for pentesters.
The tool smartly queries over 100+ OSINT sources and gathers information on emails, names, IP addresses, domains names, and more. It can also find more extensive information on a single target such as netblocks, emails, web servers, etc. SpiderFoot also understands how data is related to each other, making workflow much easier for pen-testers.
Creepy
This open-source intelligence tool collects information about geolocation by using social networking platforms and image hosting sites. Pretty creepy, isn’t it? The reports are presented on a map, as shown below, and you can filter data based on location and date. Reports can be downloaded in CSV or KML format for further studying.
Creepy is a python written tool and comes with a packaged binary for Linux distributions like Debian, Backtrack, Ubuntu, and Microsoft Windows.
Conclusion
If you want to gather reconnaissance like a pro, you should definitely have these tools under your belt since OSINT has made it easier to study organizations and networks and understand how infrastructures operate.
These tools are not only handy for reconnaissance but can be used to safeguard your network from potential threats. Whether you’re on a bug bounty or you’re just trying to maintain your network security — you should have these tools ready to use at all times.