Choosing the best editor for log search - Kote

Choosing the best editor for log search

Logs, logs, logs… they are everywhere! While scanning another endless log file, I wondered if there is any difference in the speed and performance of popular text editors when working with large log files. It turns out there is! Let’s compare Notepad, Notepad++, TextPad, and Atom in the speed of text search in log files.

Let’s focus on the following:

  • Startup time
  • Memory consumption at startup
  • Time to open an editor and a file (100 MB and 1 GB)
  • The amount of RAM used
  • Searchin for a line of text
  • Regular expression search

A few notes about apps, hardware, and testing methods

text editors:

The test was done on a laptop Dell Latitude 5511 (i7-10850H, 32 GB RAM, Windows 10). Time was measured with a stopwatch on the phone. SIP server log files were used for testing. The original file size is 100 MB, so a 1GB file was created by copying an original file ten times. Following string was used for search "340170317_134134587" and the regular expression "340170317\_\d{8}7" or '340170317\_[[:digit:]]{8}7'. The total expected amount of hits - is 33 for a 100 MB file and 330 for a 1 GB file.

Startup time and RAM consumption

Let’s start the test by comparing an empty editor window’s startup time and memory consumption.

Table 1 – Startup time and memory consumption

App

Start time (seconds)

RAM (MB)

Notepad++ 1 10
TextPad 1 8
Atom 2 367
Notepad 1 4
Note
Start time - time in seconds to open the editor 
RAM - the amount of RAM in MB displayed in the Task Manager

Startup time for most editors is less than a second. However, Atom requires a bit more time, though only two seconds in total. As for the RAM usage, Atom is noticeably ahead at launch.

Testing a 100 MB file

Initially, I planned to use the “Open with” feature in Explorer since this is how I open logs in real life. However, Atom always shows an additional info window that has to be manually closed. So the file opening time for Atom is approximate. Also, for fun, I decided to grep these files in WSL (Ubuntu) in the command line.

Table 2 – a 100 MB file search

App

Start
time
RAM

Count

Simple
Search

Regex
search

Notepad++

1

220

1

1

1

TextPad

1

43

1

1

1

Atom

4

757

1

1

Notepad

38

220

WSL (Ubuntu)

1 1
Note
Start time - time in seconds from the start of opening the file until the text appears in the UI. 
RAM - similar to Table 1. 
Count - displays only the number of matches that satisfy the search condition.
Simple Searh - time in seconds to find all hits for the string "340170317_134134587" in the entire file. The search results should display details about the matches found. For TextPad "Find in Files" function was used. Atom does not have this feature. Also, it was not tested in Notepad
Regex search - Similar to Simple Search, but using the regular expression "340170317\_\d{8}7" for search.
Search window in TextPad
Atom requires confirmation while opening a big file

Testing a 1 GB file

Table 3 – a 1 GB file search

App

Start time RAM

Count

Simple Search

Regex search

Notepad++

2

1165

4

4

8

TextPad

12

349

5

13

13

Atom

8

2619

1

2

Notepad 452

2147

WSL (Ubuntu)

5 5
Note 
Start time, RAM, Count, Simple Search, Regex search - similar to Table 2

There are a few interesting observations with the bigger file size. I am glad that after 7 minutes (452s), Notepad managed to open a file. Though, it was Not responding and freezing for the whole time. This user experience seems to be a feature 😉

Notepad slow and steady

The fastest app in terms of opening large files is Notepad++. However, it shows average results in search time. TextPad, compared to others, works slower, but it is not hungry for the RAM simultaneously. Atom is blazing fast in search, but make sure you have enough RAM to feed it.

Search window in Atom. Count (found) displays results blazing fast

Find in Files – directory search

And the final test is Find in Files. The directory contains ten files of 100 MB each. The total volume is 1 GB.

Table 4 – Find in Files

App

RAM

Simple Search

Regex search

Notepad++

220

6

11

TextPad

14

13

14

Atom

401

6

6

WSL (Ubuntu)

6

6

Note
RAM,Simple Search, Regex search - similar to Table 2

In this scenario, communication with the file system is more important than a text editor.

Atom Find in Project (Files) results

To sum up

Different applications work differently with text log files. It becomes especially noticeable on large files or when using the Find in Files feature.

It looks like Notepad reads the file in one thread, so it takes forever. It also requires a lot of RAM; as a bonus, the interface does not respond during the search.

Notepad++ loads file quickly, and search is fast, but it also consumes memory proportional to the file size.

TextPad doesn’t have high requirements for the RAM, though it takes much longer to open files (obviously to convert them).

 Atom is not the fastest opening files, it needs a few times more memory than the file size, but the search time for opened files is the best.

All in all, it seems that the best editor has not been released yet. Each of the apps on the test solves problems in its way. On the other hand, it is worth mentioning that the standard Notepad is not suitable for working with logs at all.