[New Plugin] MultiReplace

guy038

First, note that I use your last version 6.0.0.36 !

In this post, I did tests about the Search in Files feature with the three N++, Search and MultiReplace features.

Globaly, they all give the same results with tiny differences regarding the MultiReplace plugin, which will require a follow-up inspection

Now, in my last post to @coises, I did similar tests and this time, the MultiReplace plugin give completely different results, compared to native N++ and Search++ ! Indeed, I got :

27,081 matches in 841 files of 1,772, in 2m 10s, with Search++
27,081 matches in 841 files of 1,772, in 1m 21s, with N++

But only :

18,767 hits in 726 files in 1m 05s, with MultiReplace

As I said, in @coises’s post, I first suspected your File Search : Skip files larger than option, in Settings, to be enabled but it was not set !

Here is a screenshot of the MultiReplace dialog and of the Search Results panel :

So, did I do something wrong ?

I also did a test on an other USB key where I put a simple subset of 7 files, from my D: key, in a Test folder
I moved, within a command prompt, to that Test folder
I started N++ without a session and opened these 7 files only
I first ran the Find in Files operation, with the Directory option = E:\Test and the Filter option = *.* => 83 hits in 3 files
Then, I ran the similar Find in Docs operation => 372 hits in 7 files !

So, an obvious difference ! If necessary, I could send you the contents of this Test folder, for further tests !

Best Regards,

guy038

Thomas Knoefel

@guy038
Hello @guy038,

Thanks for this thorough test, and no, you didn’t do anything wrong. Your investigation uncovered two separate behaviors in the code:

Binary Files: MultiReplace “Find in Files” currently skips binary files, while native N++ searches every file it lists. That accounts for most of the gap in your USB drive numbers. I’ll look at how to make this configurable and more transparent.
UTF-16 Bug: There is indeed a bug in how “Find in Files” handles UTF-16 encodings compared to “Find in Docs” and “Replace in Files”. This most likely explains the missing files in your smaller Test folder too.

Since the forum isn’t ideal for sharing files and tracking bug fixes, I have opened a GitHub issue for this here: https://github.com/daddel80/notepadpp-multireplace/issues/133
I would gladly take you up on your offer to share the E:\Test folder contents so I can verify the fix against them. Could you please attach that folder as a zip or provide a link in the GitHub issue?

Thanks again for your time and for helping to polish the next release candidate!

Thomas Knoefel

MultiReplace 6.1 is out and will land in Plugin Admin with the next Notepad++ release.

The focus of this release is Replace All in One Pass, running the whole list in a single left-to-right sweep instead of one full pass per entry, together with a reworked Find/Replace in Files and a handful of list and UI improvements. Several new options live under Plugins > MultiReplace > Settings.

Download: Latest build

Key Features & Improvements

Replace All in One Pass: Runs the whole list in a single pass instead of one full Replace All per entry, so results build on each other left to right. With cat → dog and dog → cat, sequential mode turns “cat dog” into “cat cat”, one pass turns it into “dog cat”. Much faster on large documents. Toggle per tab, right-click the list tab.
Use Open Documents: Find/Replace in Files can now work on files already open in Notepad++ as the live document, unsaved changes included, instead of the copy on disk. New checkbox in Settings.
Skip Binary Files: New option for Find/Replace in Files (on by default), same detection as ripgrep and VS Code. Also adds UTF-16 detection without a BOM, and skipped files are always reported, broken down by reason.
CSV Header Rows: Find/Replace now always searches header rows too. “Header rows to exclude” only protects sorting and duplicate detection.
Collapse to File List: New Search Results dock option folding every search block down to just file names and hit counts. Ctrl/Shift-click works in the fold margin, same as Notepad++'s own Search Results panel.
Library Mode: “Keep list always visible” can now be switched right from the Use List button’s context menu, no detour through Settings.
Add New Row: Takes over the dialog’s current search options instead of starting all off. Find/Replace text still starts empty.
Duplicate Row Reporting: A list row that exactly duplicates an earlier enabled one is skipped as before, but the status line now tells you how many rows were skipped.

Issues or suggestions: GitHub.

Happy replacing!

Thomas Knoefel

MultiReplace 6.2 is out and will land in Plugin Admin with the next Notepad++ release.

This is a big review and performance release: Find/Replace in Files is considerably faster and more robust, and the existing features were gone through once more, which led to a long list of fixes.

Download: Latest build

Key Features & Improvements

Faster Find/Replace in Files: Files are now read on worker threads while the current one is searched. In a test with 48,500 files, a search dropped from 8.7 s to 6.0 s. The thread count follows the drive type and can be set with LoadThreads in the INI file.
Faster scan and results: Files that cannot contain a hit are ruled out right after reading, the folder scan was reworked, and large result sets appear much faster in the Search Results Window.
Robust runs: Unreadable folders no longer end a scan; they are skipped and reported, for example 3 folder(s) unreadable. The panel is locked during a run (Cancel stays available), and a file that changed on disk after it was read ahead is read again, so nothing gets overwritten.
Safer Replace in Files: Files it cannot process safely are left unchanged and reported with the reason, for example 2 file(s) skipped: 1 too large, 1 not writable. “Not writable” is new, for example a file held open by another program.
Fixes from the review: Closing Notepad++ during a run no longer crashes. Find/Replace in Open Documents returns to the tab you started from and keeps its Flow Tabs alignment. With the CSV scope, each document uses its own columns. A hit in an unsaved tab (“new 1”) now jumps to that tab.
Translations: Japanese and Traditional Chinese show hit and file counts in the right order, Chinese (Simplified) can copy and open paths from the Search Results Window again, and the progress text is translated.

All changes are in the release notes.

Issues or suggestions: GitHub

Happy replacing!

guy038

Hello, @thomas-knoefel and All,

Again I made a comparison between the Notepad++ v8.9.8, Search++ v0.72 and MultiReplace v6.2 Search in Files feature against all files of my D USB key.

All the Search results panels return 27,053 hits in 646 files of 1,153 searched, obtained in :

1m 43s for Notepad++
1m 13s for MultiReplace
2m 18s for Search++

As previously, I unpluged my USB key, restarted my computer and repluged my key before any Search in Files action

Your list of improvements, in your latest relesase, are quite impressive ! And probably, my test does not exploit all the features of your plugin. I still need to carefully read your documentation !

Best Regards,

guy038

Thomas Knoefel

Hi @guy038,

thanks for testing again! In case you’re curious, here’s roughly what the new loading threads in 6.2 do.

As soon as the file list is known, a few worker threads start reading ahead: they open the files, read them, check for binary content and decode the text. The main thread takes the files in list order as soon as each one is ready, puts it into a hidden Scintilla document and searches it. So the hits and their order stay the same, but reading and searching now overlap.

On a USB stick MultiReplace uses four threads by default. With a cold cache, as in your test, most of the time goes into waiting for your USB key, and that is where it helps: up to four files are read at the same time, and the search runs while the next ones are still loading.

If you want to see what the threads bring on your key: with LoadThreads=0 in the [ReplaceInFiles] section of the INI file, every file is read on the main thread as before (see the README; it takes effect after restarting Notepad++).

guy038

Hello, @thomas-knoefel and All,

Indeed, changing the line :

LoadThreads="auto"

into this one :

LoadThreads=0

within the MultiReplace.ini file, the same list of files were found in 2 m ( instead of 1 m 13 s )

Thus, I decided of an experiment, changing, on purpose, the LoadThreads line to :

LoadThreads=8

After restarting my computer and doing the test, the execution time was, this time, 1 m 28 s. However, verifying within the MultiReplace.ini file, I saw the line :

LoadThreads="0"

Is this the expected behavior, with the number 0 as a string ?

Best Regards,

guy038

Thomas Knoefel

Hi guy038,

interesting series of runs, thanks! Both runs without threads, the 2 m one and most likely the 1 m 28 s one too (see below), are slower than the 1 m 13 s with auto. So the threads do help on your key, even if cold runs vary quite a bit.

Yes, the quotes are expected. LoadThreads is stored as text because it can also be "auto", and the quotes are removed when the file is read, so 0 and "0" mean the same.

What changed your 8 back to 0 is something else: MultiReplace reads its INI file once at startup and, once its panel has been opened, writes all its settings back when Notepad++ closes. An edit made in the meantime is therefore overwritten on exit. So your 1 m 28 s run most likely still used LoadThreads=0.

To test 8: change the line either in Notepad++ before opening MultiReplace, or with another editor while Notepad++ is closed. Then restart Notepad++, and the new value is used.

Curiously, in my experience with other tools, four to six parallel threads were always the optimum.

guy038

Hi, @thomas-knoefel and All,

For these Search Results : Search "Fi" (27 010 hits in 646 file(s)) [1 153 file(s) searched], I got, successively, using your valuable instructions :

1 m 13 s 78, then 1 m 10 s 85 and finally 1 m 11 s 06 when LoadThreads="auto"
1 m 25 s 78 and 1 m 26 s 39 when LoadThreads="0"
1 m 09 s 54 and 1 m 09 s 55 when LoadThreads="8"

So, the two tests with LoadThreads="8" seem very slightly better !?

Note that :

From Laptop documentation, my processor is an Intel® Core™ i7-1165G7 2.80 Ghz (up to 4.7 GHz with Intel® Turbo Boost Technology ) 12 MB L3 cache, 4 cores 8 logical processors
From informations système, I saw for Processeur : 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz, 2803 MHz, 4 cœur(s), 8 processeur(s) logique(s)
Total installed RAM : 32 Gb

This explain why I wanted to test with 8 threads. But, may be this assumption is not pertinent ! ?

Best Regards,

guy038

Thomas Knoefel

Hi @guy038 ,

to your question about the 8 logical processors: they don’t really matter here. The load threads do very little computing, they mostly wait for the key to deliver the next file. How many reads can usefully run at once depends on the drive, not on the CPU. That’s why auto picks the count by drive type: 4 for local drives like your key, 8 for network drives, where every file waits for the server, and 1 for hard disks, where parallel reads would just make the heads jump back and forth. On a key with a weaker controller, 8 could just as well be slower than 4. On yours it’s slightly faster, so feel free to keep it. 8 is also the maximum.

One more thing: your first comparison showed 27 053 hits in the same 646 files, now it’s 27 010. Did anything change on the key in between?