Search++: A work in progress

guy038

Hello, @coises and All,

Your tests are much more comprehensive than my simple test and yield some interesting results !

In particular, the comparison between Notepad++ and Search++ for searches on a USB drive shows that Search++ outperforms Notepad++, whether the search is complex or simple !

Indeed, we have :

Search++ 0.6.3.1 :

U:\Testing2           a([^\r\n])[^\r\n]*b\1  0:16.36  149,919 matches in 2,505 of 3,762 files (Regex): a([^\r\n])[^\r\n]*b\1
U:\Testing2 (repeat)  a([^\r\n])[^\r\n]*b\1  0:08.87  149,919 matches in 2,505 of 3,762 files (Regex): a([^\r\n])[^\r\n]*b\1

U:\Testing2           the                    0:24.84  445,907 matches in 3,342 of 3,762 files (Regex): the
U:\Testing2 (repeat)  the                    0:01.03  445,907 matches in 3,342 of 3,762 files (Regex): the

And :

Notepad++ 8.9.7 :

U:\Testing2           a([^\r\n])[^\r\n]*b\1  2:08.20  Search "a([^\r\n])[^\r\n]*b\1" (149919 hits in 2505 files of 3762 searched) [RegEx]
U:\Testing2 (repeat)  a([^\r\n])[^\r\n]*b\1  1:54.96  Search "a([^\r\n])[^\r\n]*b\1" (149919 hits in 2505 files of 3762 searched) [RegEx]

U:\Testing2           the                    0:31.62  Search "the" (445907 hits in 3342 files of 3762 searched) [RegEx]
U:\Testing2 (repeat)  the                    0:14.16  Search "the" (445907 hits in 3342 files of 3762 searched) [RegEx]

I’m actually really surprised by Search++'s performance compared to Notepad++, especially when the search involves a complex regular expression. ( 0:16.36 vs 2:08.20 ! )

Please, note that the USB flash drive I used for my test is a fairly old one with a capacity of only 512 Mb, which may not be well-suited for multithreading and would explain the poor results with Search++

In any case, I assume you’re now confident in Search++'s overall performance !

Best Regards,

guy038

Coises

@guy038 said:

I’m actually really surprised by Search++'s performance compared to Notepad++, especially when the search involves a complex regular expression. ( 0:16.36 vs 2:08.20 ! )

Please, note that the USB flash drive I used for my test is a fairly old one with a capacity of only 512 Mb, which may not be well-suited for multithreading and would explain the poor results with Search++

In any case, I assume you’re now confident in Search++'s overall performance !

I don’t entirely trust my own results. Why did my USB search in Search++ for the take longer than the search for a([^\r\n])[^\r\n]*b\1? Either the test data is erroneous or spurious, or Search++ is screwing up access to the USB in some way that’s causing it to “choke” and lose performance (what I suspect happened in your test). I’m still trying to work out how to avoid that happening without hurting the performance when reads are fast.

To the best of my knowledge, Notepad++ is entirely single-threaded. It’s only ever doing one thing at a time. An i9-9900K can run sixteen threads simultaneously. So for CPU-heavy workloads (complicated regex searches) with lots of files, it’s not at all surprising that a multithreaded approach could be almost eight times faster. Thirteen times faster running a([^\r\n])[^\r\n]*b\1 against an SSD is pretty satisfying, though. :-) I’m probably getting additional gains from using memory-mapped I/O and running Boost::regex directly against the data the operating system maps into memory, while Notepad++ loads every file into a buffer, and from there into an off-screen Scintilla, before running the search through Scintilla’s API instead of directly with Boost::regex.

It’s useful to have tests like yours. I won’t be able to test anywhere near all the situations that can occur. What happens when someone searches a OneDrive folder and not all the files are up-to-date? What happens when someone searches a drive that’s mounted over a VPN? What happens when the cat knocks the USB connector loose in the middle of a search? The worst case for all these is that the whole thing either crashes and takes Notepad++ down with it, or locks up and makes the user terminate Notepad++ with task manager — losing any unsaved work in progress. I don’t want to let that happen. I still have a lot to do to “harden” this against things that can go wrong.

And that’s not even starting with Replace yet.

sevem47

First of all thanks to your great work with this plugin. Going through the help file to learn about all features of the plugin the following ideas came to my mind.

They are about graphical topics that probably are just cosmetical questions.

Using a docking dialog for different plugins they may share the same space in NPP. If Search++ ist not active, there is just an empty space that is visible for me. Other plugins have a graphical representation that, in addition to the balloon tip, show which plugin is active. Two images to show my point:
/
Using Search++ I learned about the possibility to show only certain lines in the edit window. Would it make sense to have a marker that identifies that only some lines of a document are displayed after applying Search++ commands. Something like the following filter symbol to mark that not all lines are visible:

(the image is copied from Total Commander’s function ‘Synchronize directories’)

Just some ideas that would help me in using Search++.

Coises

@sevem47 said:

First of all thanks to your great work with this plugin.

Thank you for trying it and giving feedback!

Using a docking dialog for different plugins they may share the same space in NPP. If Search++ ist not active, there is just an empty space that is visible for me. Other plugins have a graphical representation that, in addition to the balloon tip, show which plugin is active.

Good point. I will fix that — not necessarily in the very next release, but I will fix it.

Using Search++ I learned about the possibility to show only certain lines in the edit window. Would it make sense to have a marker that identifies that only some lines of a document are displayed after applying Search++ commands.

I might be missing something. Do you find that it is not obvious enough in the document window itself when lines are hidden (the horizontal lines where the hidden lines are)?

sevem47

@Coises said:

I might be missing something. Do you find that it is not obvious enough in the document window itself when lines are hidden (the horizontal lines where the hidden lines are)?

Thanks for your hint; I see, the mistake is on my side! I use a different background color and my eyes are getting older ;-) Therefore I did not see the lines between the text:
My configuration vs. Default theme:
/
Now I know where to look.

Good point. I will fix that — not necessarily in the very next release, but I will fix it.

I know you have a lot of topics on your list that are much more urgent. For me it is a ‘nice to have’. Thanks for your help.

M Andre Z Eckenrode

This post is deleted!

M Andre Z Eckenrode

@Coises

As you know, I recently reported an issue with Search++ in a new, separate topic, but I see that several others have reported various issues in this one (presumably, because it’s still a work in progress). I have a new issue to report, and thought maybe I should ask if you prefer that new issues get reported in their own topic (which is generally my default), or for users to just include them in this one. Do you care one way or the other?

Coises

@M-Andre-Z-Eckenrode said:

@Coises

As you I recently reported an issue with Search++ in a new, separate topic, but I see that several others have reported various issues in this one (presumably, because it’s still a work in progress). I have a new issue to report, and thought maybe I should ask if you prefer that new issues get reported in their own topic (which is generally my default), or for users to just include them in this one. Do you care one way or the other?

Either way is OK with me. I suspect the moderators might prefer to keep discussion confined to this topic, though.

Coises

Search++ version 0.6.4 is available. It is expected to fix the bug reported here.

Even though it seems most people never experience that bug, I still recommend upgrading at the next convenient time, so we will all be testing the same code. It’s always possible that I broke something else.

I am still working on improving the multi-threading for find in files. I’m also trying to decide how best to mitigate a problem wherein Search++ fails to load when a particular Windows redistributable library has never been installed on the system. More updates should be coming before too much longer.

All thoughts, suggestions, comments, criticisms, bug reports and raspberries are welcome, and thank you to everyone willing to give this jalopy a good work-out.

Coises

Search++ version 0.6.5 is available:

Update to ICU 78.3 and use static linking for ICU. This is expected to fix problems some testers have had running Search++ on older systems or on minimal Windows installs (like Windows Sandbox). For most users there will be no functional difference between this version and version 0.6.4.

Note: There are five icu——.dll files in older versions of Search++ that are not used beginning with version 0.6.5; so it is best to delete the existing Search++ folder (or its contents) before copying, rather than copying the new folder over the old one.