Beginners guide to Log File analysis in 2025

Log file analysis was one of the primary SEO techniques during the early years of search and digital marketing, but the rise of various SaaS products has led to it becoming a dying art. However, there are still insights that can be gained from analysis of your raw access logs and we hope to provide you the information you need to decide whether it’s a worthwhile investment of your time and how to conduct it if it may be.

What is log file analysis?

Log file analysis is the process of either manually, or using a tool or platform, reviewing the data that is stored by your site’s servers whenever a request for a resource (web page, CSS/JS file, image etc.) is registered. In doing so, the analyser can reveal issues with various parts of the site, possible SEO opportunities and the general behaviour of various search engine crawlers that roam the web.

What do log files include?

While log files can be extremely useful for SEO, the information stored is pretty basic – this includes:

• HTTP Status Codes: Identifies responses such as successful loads (2XX), redirects (3XX), and errors (4XX/5XX).
• IP Address: Tracks the source of requests.
• Request Method: Indicates GET (retrieve data) or POST (submit data) requests.
• Timestamp: Logs the date and time of each request.
• URL Path: Shows the resource being accessed.
• User Agent: Specifies the browser, bot, or crawler making the request.
• Referrer URL: Helps identify where the request originated.
• Response Time: Measures server performance for each request.

These files will vary in size depending on how large the site is, how much traffic the site gets and how regularly the logs are archived.

Where can you find your site’s log files?

The only servers I have access to at the point of writing use the cPanel GUI, but most are fairly similar, so while they may not be in the same place, they’ll generally have the same descriptions. So, to find your log files, you’ll need to access your server management platform or, if you use a CDN, you’ll find your logs there as the server won’t receive most of the data.

Once you’re in, you’ll have two potential options, you can select ‘File Manager’ and scroll down to the ‘Logs’ folder, or you can select the Raw Access option which will allow you to download the most recent files.

You’ll receive a zip file (gz or similar) which you can unzip with Winrar to allow you to open it in your preferred spreadsheet program or SaaS log file analyser. If you’re opening in a spreadsheet, however, you’ll generally need to separate the text into columns as it will generally paste into a single cell per hit.

If you’re in Excel, you can do this with the ‘Data’ tab, and the ‘Text to Columns’ option.

The result should be columns which will fit into the following structure (sometimes there are a couple more, sometimes a couple less):

• IP Address
• D/M/Y/HH:MM:SS
• Method/Query
• HTTP Status Code
• File Size/Bytes Downloaded
• User Agent