Change logs

===============================================================================
Change log for version 2.3
===============================================================================

Fixes:

( 1) Fixed a problem with over-zealous treatment of what is “hidden text”, e.g. https://jamiemetzl.com/origins-of-sars-cov-2.

( 2) Added a missing automatic refresh when a web query is canceled.

( 3) Fixed a typo which caused a data type mismatch when modifying the buffer offset value

( 4) Fixed a problem where some url images were incorrectly perceived as duplicates and therefore removed from the text when converting to HTML using Copy and Edit.

( 5) When removing a file which is not the last, the subsequent file would take on the font characteristics of the removed file. This has been fixed.

( 6) The context menu entry “Unset filter, then find selected text” now works correctly

( 7) Added support for urls using the HTTP protocol which only work with HTTPS

( 8) Fixed a problem where tables inside of figure tags were being ignored

( 9) Fixed a problem where multiple versions of the same image with different resolutions were treated as different images during Copy and Edit

(10) Fixed a problem where hyphenated tags were not being processed correctly.

(11) Fixed a number of problems which caused incorrect table and blockquote formatting in certain cases.

(12) Urls containing a subdirectory named “audio” (case-insensitive), e.g., https://www.xyz.com/audio/src will be treated as downloadable only audio files despite not having an audio extension.

(13) Fixed responsiveness of filter toggle button.

(14) Fixed the case where hidden text information at the top of the file would prevent ^s from using the web page title as the subject of the email.

(15) Fixed a problem where some images would not be rendered in Outlook when converting to HTML using Copy and Edit.

(16) Fixed a problem where the wrong data may be used when dropping text which contains embedded file names but no urls.

(17) Fixed a couple of problems related to doing a file search via a right-click of “add Files” when minimizing the primary window during a long search.

(18) Fixed a problem with the sync’ing of the (pin) on-top button and the actual on-top status.

(19) Fixed a serious bug which could cause the program to freeze when pasted text contained quoted strings. Multiple urls and/or files may be pasted or dropped into the viewer to be processed. However, if more than one is one a single line then all such items MUST be quoted. For example,

“http://time.com”
“http://www.rawstory.com”
“http://www.livescience.com” “http://allsides.com”

will have all of the urls recognized but

“http://time.com”
http://www.rawstory.com
“http://www.livescience.com” “http://allsides.com”

may not. 

(20) Fixed a problem when deleting urls from the url history list. If delete was pressed but no urls had been selected and there was a warning message to this effect that message would preclude any further deletions from happening until the program was restarted.

(21) Fixed a problem where urls containing spaces were considered invalid when attempting to load them from a ‘list file’, viz., the list of pre-read files to be reloaded in a batch.

(22) Fixed a problem where urls containing “#” could cause filtering to get out of sync and generate beeps when the cursor was over an “apparently” bad hyperlink.

 

Changes:

( 1) The default download filename now uses the rightmost two url fields to create a possibly more meaningful name.

( 2) Added the right-click option to the “t” command on the button bar to make it equivalent to the “Pin on top” button, i.e., go to the bottom of the file and, if there already, go to the top.

( 3) Next page processing is now always attempted when the host name of the original url and the potential next one to be read are identical. Previously, if the host names were the same but the 2 urls differed substantially the next page was ignored, e.g., https://leadstories.com/hoax-alert/

Plus other miscellaneous fixes

===============================================================================
Change log for version 2.2
===============================================================================

Fixes:

( 1) Requesting unique data lines when extracting text using matching could cause an error:
“This key is already associated with an element of this collection”

( 2) When using Copy and edit to back create HMTL
– Added support for image urls embedded in a primary url
– Added support for certain image urls containing embedded formatting information

( 3) When attempting to open multiple urls/files if two or more were identical then some or all of the selected urls/files would be ignored

( 4) Fixed a problem where the Esc key was not being recognized during a multifile find operation

( 5) Fixed a problem with certain HTML table renditions

( 6) Fixed a problem where some hyperlinks were incorrect

( 7) Image processing for ‘Copy and edit’ has been improved and a number of problems were fixed particularly when choosing to display 2-5 or all images per line when only image urls are selected

( 8) Removed spurious CAPTION, etc. text from figures which can’t be rendered by RCFV

( 9) Fixed a problem where urls containing an embedded ‘https://’ such as ‘https://archive.is/https://www.wsj.com/articles/…’ were incorrectly modified

(10) Fixed a problem where certain url’s ‘next page value’ was not calculated correctly because the host name looked like downloadable data

(11) Fixed {Ctrl}+{d} (duplicate line) and {Ctrl}+{l} (delete line) code in the scratchpad

(12) Significantly improved the code which was slowing down matching of urls against the url history list

(13) Fixed a number of issues related to pasting and processing HTML text

(14) Fixed a longstanding issue where, if multiple lines of text were selected which contained a hyperlink, the left click popup menu would be invoked.

(15) Fixed a problem when dropping or pasting multiple html files which prevented the user from viewing them as literal text or processing the html text

(16) When pasting HTML text containing urls, the option to reformat the urls to a separate line was not provided unless there were urls preceded by a space or a tab. Now any url found in the text not already starting on a new line will trigger the option.

(17) Fixed a couple of bugs related to urls which end with “/?”

(18) Fixed responsiveness of {Esc} key when attempting to open multiple files

(19) Fixed a problem where null characters, \x00, were being deleted which affected the correctness of the hex viewer

New features:

( 1) Added support for Flipboard entries

( 2) Added the ability to explicitly set the RGB value for the filter color values via a right-click

( 3) You can now automatically append “^^” and “$$” to the beginning of a SINGLE line of filter text which is dragged from the viewer to the include filter. Hold down the Ctrl key while dragging to prepend “^^” to the dragged text or the Alt key to prepend “$$”. This makes it easier to create Start-End expressions. If you use this for the exclude filter the Start-End prefixes, viz., “^^” and “$$”, will not be appended (they are invalid in the exclude filter) but the selected line will automatically be added to the filter.

( 4) Some sites store the text to display as part of a JSON structure. RCFV now extracts that text enabling you to read the essence of the web page albeit without ideal formatting, e.g., https://www.scmp.com/news/china/science/article/3205869/why-do-chinas-mild-covid-symptoms-feel-so-unbearable. Note that to view JSON text you must enable the “Show hidden text” option. To improve the readability of some JSON text we suggest you also check the ‘Fix up run-on text’ web page option.

( 5) Hidden text processing has been improved a bit. If the “Show hidden text” option is not specified (which is the norm) then when there is a non-trivial amount of “significant” hidden text, viz., greater than 20% or so of the total text, you will be notified at the top of the document. (Note that hidden text will no longer be displayed automatically if there is little other text available to display on a web page.)

( 6) Double-clicking while the {Ctrl} key is depressed will select the “paragraph”, viz., all text between the preceding linefeed and the subsequent carriage-return.

( 7) Display of and captioning for web images using ‘Copy and edit’ has been improved.

( 8) Using ‘Copy and edit’ when your selection contains only image links or files you have the option of displaying them in a columnar format with all on one line or in 2-5 columns (N-up). When you do this the default image width is normally used. You now have the option of overriding the width value to force all of the images to have the specified width and fit perfectly in each column. Note that this may cause some images to be distorted. You can use this to constrain the width of other images as well although this is probably less useful.

( 9) You can now use the {m} quick-key while in the viewer to convert the current display font to monospaced, viz., Courier New.

(10) When dragging text from the viewer to a filter box regex special characters will always be escaped (“\” prepended). However, when dragging text from the scratchpad or other source you may not want to escape the text because, for example, it is already escaped or it is a well-formed regular expression. You now will be asked if you want to have it escaped automatically if it contains an unescaped regular expression special character.

Changes:

( 1) The {F12} function, “drag to window”, which allowed you to create an on-top small window to drag data to is no longer supported. It is unnecessary since you can just do the same to the taskbar icon.

( 2) Modifed the context menu to indicate that when a file or files are selected you have the option of opening them in their associated application which is not necessarily the default browser.

( 3) If the filters are otherwise empty except for a $^F4 (auto-advance to first text block) filtering will be turned off.

( 4) Pressing the ‘Save filters’ button for a url without a matching fvf file will use the url value as the template for the default fvf file name instead of ‘Default.fvf’ if the Default.fvf file exists.

( 5) RCFV typically looks for embedded urls and/or files in pasted text and gives you the option of opening those instead of the containing text. This can take a little time for large files so it has been removed for files larger than 20,000 characters or having greater than 1000 lines. You can always create a filter for those objects and extract them from the larger text if that’s what you want to do. Since this is probably atypical it was decided that the performance penalty wasn’t worth the slight benefit.

( 6) Drastically improved the performance when deleting multiple urls while in url name order

( 7) When dragging and dropping or pasting MULTIPLE lines to the exclude filter input box, if at least one of those lines is “blank” (only contains 0 or more space characters), it may cause the display to become less legible considering that one or more spacing lines will be filtered out. In this case the blank line expressions will NOT be included in the filter list to retain proper spacing for the document. Hint: make sure that the RCFV special expression “$^ *” is included in the exclude filter to remove extraneous blank lines.

( 8) Some filter updates were made but, most importantly, recent (Google) news was corrected, to wit,

Include filters: Case Insensitive
\d+ (minutes|hours) ago 20\d\d(?=-)
Yesterday\s+20\d\d(?=-)
\d+ days ago 20\d\d(?=-)
$^TM3.+20$^yy(?=-)
$^PM3 [23]\d.+20\d\d(?=-)


Exclude filters: Case Insensitive
$^—\d\d-\d\dT\d\d:\d\d:\d\dZ
$^–(?<=ago)\s+20\d\d
(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct).+20$^yy-1
—————-
Context[3,0]

Notes:

( 1) svg image files are not currently renderable by ‘Copy and edit’

Plus other miscellaneous bug fixes

===============================================================================
Change log for version 2.0
===============================================================================

New features:

(1) Added a new quick lookup feature. Using {Ctrl}+{F2} optionally including the {Shift} or {Alt} keys followed by a single alphanumeric character (within a few seconds) such that that alphanumeric character may be used to quickly select one or more lookup list urls. Each item description may optionally be decorated at the beginning with one or more alphanumeric characters followed by a colon. When the character selected by the keystroke following {Ctrl}+{F2} or one of the other variants matches one of those “decorations” the list item will be selected.

Note that the match is case-insensitive, viz., “Q” will match “q”.

{Ctrl}+{F2} then keystroke ==> use tbe browser to open the matched urls (see below)
{Ctrl}+{Shift}+{F2} then keystroke ==> open the matched urls in RCFV (with the option to open in the browser instead)
{Ctrl}+{Alt}+{F2} then keystroke ==> append the matched urls to the scratchpad

For example, if the list items are as follows:

[q:Qwant] https://www.qwant.com/?client=ext-chrome-sb&q=??&t=web
[g2:Google] https://www.google.com/search?q=??&num=100
[b2:Bing] https://www.bing.com/search?q=??
[i:Bing images ***] https://www.bing.com/images/search?q=?? ;***: USE BROWSER ONLY

“g” following {Ctrl}+{F2}, etc. selects Google, “b” selects Bing, and “2” selects both

( 2) All variants of lookup may now be done from the scratchpad as well using whatever text is highlighted or the current word, if any.

( 3) Added the ability to right-click a normal filter button highlight color to be able to copy that color to all subsequent buttons to its right

( 4) Added an option to “Copy and edit” to copy the simplified HTML of the viewer text directly to the clipboard obviating the need to copy the HTML from a browser tab

( 5) Added an unescape function for selected text in the scratchpad to deal with “encoded” urls such as
https:\/\/chrt.fm\/track\/15E3G4\/traffic.megaphone.fm\/GLT9156179404.mp3

It will also “decode” urls which contain Unicode constants for all ASCII characters, viz., “\u0000” through “\u00ff” such as “https:\u002F\u002Fwww.stitcher.com” => “https:\\www.stitcher.com“

( 6) Added the ability to use F7 (paste special RCFV control characters) in the scratch pad

( 7) Added support for the tab key in the scratchpad

( 8) Added support for column selection where the columns are delimited by multiple spaces, e.g., $^2\s will select the third column where the columns are delimited by one or more spaces

( 9) {Alt}+{/} may be used anywhere to toggle the control panel and make the viewer the active control. Note that the  {/} key alone may only be used to do this when the viewer is the active control.

(10) Updated videos for early instances of version 2 and added a new video: Extracting Columns From A Csv File

(11) Added the ability to select only folder names for a path list when selecting “files” (right-click ‘add files’)

(12) Added the option to extract only unique lines of data when right-clicking the Refresh button (match extraction)

(13) Added the option to include date and file and folder size attributes in path lists

(14) Extra control has been added to Bulk File Selection (right-click add Files) such that one can ignore non-text files dynamically for each selection and also ignore hidden files and all files, hidden or not, which are located in hidden folders

(15) Added a marker character, ‘†‘, after the link descriptor text when the urls are moved below the text when using Copy and edit to paste text with links to the clipboard

(16) Pressing the Escape key in the viewer may be used to (1) stop long-running operations, (2) cancel most dialog boxes, (3) unselect any highlighted text and (4) pause auto-scrolling

Changes:

( 1) Deepl translation as a Lookup feature has been removed. You can download their free translate app which works in a similar way within any program at https://www.deepl.com/app. It has been replaced in RCFV by Reverso translation. For existing users the changes appear in ‘LookupText.install.txt‘ in the Data folder.

( 2) After an fvf file was matched by a url the file name used to be copied to the clipboard. That is no longer the case: the clipboard will now remain undisturbed by an fvf file match. One can interrogate the name of the matching fvf file by pressing {Alt}+{v} which will append that file name to the scratchpad.

Fixes:

( 1) Fixed FastReplace error due to unsorted inputs

( 2) Fixed problem with “$” regular expression characther when used in a Find {F3}

( 3) Modified the code which inserts “^” and/or “$” in strings dragged from the viewer to the Find input box or one of the filter input boxes

( 4) Fixed lack of removal of “+-” context character prefix when converting to simplified HTML

( 5) Fixed a regression where extended regular expressions starting with “$^::“, viz., match the underlying url value of a standard hyperlink instead of the link text, was not working

( 6) Fixed an issue which prevented HTML text from being processed in RCFV directly

( 7) Fixed lack of support for \t as a delimiter character for column selection, e.g., $^2\t will select the third column where the columns are delimited by tabs

( 8) Fixed a problem where the length of MS Office files was calculated incorrectly since it used the actual length of the source file but not the length of the (automatically) converted text file which is displayed

( 9) Fixed an issue where a new fvf file was unable to be saved due to an invalid file name

(10) Fixed a problem where very small web pages were not displayed properly

(11) Added support for text lines delimited solely by a carriage-return, x’0d’

(12) Fixed a problem where a single line dropped into the include or exclude list input box would be automatically added to the list box

(13) Fixed a bug where a negative regular expression for selecting files (right-click ‘add files’) would only work for the top level folder

(14) Fixed a problem which occurred when changing the “intra-line matched text delimiter” in the General Settings screen

(15) When using {Ctrl}+{s} to send an email containing the current url(s) the redirected url name, if any, is now used instead of the original url. This will also affect other places where the url name is displayed such as the file list.

(16) Fixed a problem where downloadable urls were not being recognized correctly. This issue also suppressed the inclusion of some links which were incorrectly considered duplicates.

(17) Fixed bad reformatting which caused spaces before punctuation followed by a digit to be elided

(18) Fixed a problem where HTML tags inside of <pre>…</pre> tags were not handled

(19) Fixed a number of issues related to processing multiple files, especially list files (files which start with ‘!’), and the creation of a path file (a single file containing all selected file path names)

(20) Fixed a number of problems related to the number of selected files for a file path

(21) Fixed url copy (right-click 7-Export | Copy selected urls) in the url list (internet command) when ordered by url value

(22) Fixed “Find local label” (an anchor beginning with “#”) so that it actually finds it if it exists in the text

(23) Fixed an issue with file lists where the saved information, e.g., font, colors, etc. was not being applied to the files in the list when they are opened from a file list

(24) A number of Bulk File Selection (right-click add Files) issues were fixed and/or improved.

(25) Fixed highlighting when a column specification, e.g., $^4\s was ANDed with a normal expression.

(26) When a single file is added to the viewer it is assumed that a path list is not wanted

plus other minor bug fixes

===============================================================================
Change log for version 1.2
===============================================================================

( 1) Improved recognition of images in files when they are specified as SRCSETs.
( 2) Added support for urls which contain additional ‘//’ strings after the protocol, e.g., https://www.the-scientist.com//?page=2
( 3) Fixed a bug where captured lines containing a backslash, ‘\’, included some spurious characters
( 4) Fixed anchor formatting in superscripts
( 5) Fixed additional HTML formatting issues for back conversion
( 6) Added a new extended regular expression to simplify undelimited column highlighting
– e.g., $^10:5 will match the five characters in every line starting at the 11th column
( 7) Modified and clarified support for drag and drop vs. copy and paste for text added to filters
– you can now copy and paste multiple lines into the filter input box
– the two modes differ only when the source of the text is from the RCFV viewer screen
– drag and drop will escape any regex characters in the string and optionally add
some extra characters such as “^” and/or “$” to make the match more precise
( 8) Modified the algorithm for matching fvf file names such that if there are multiple matches
the file matching the longest string of characters in the url will be used
– if there are multiple such file “expressions” that fit that definition of “best match” then
the longest file name of those will be used
– if there are multiples of those then the first one encountered will be used
( 9) Added the ability for a user defined default fvf file (named, of course, “DEFAULT.fvf”) which,
if it exists, will be used when no other match is found. In the most minimal case it can be used
to define just the default color scheme.
(10) Added support for “msn encoded” news urls
(11) Added support for urls in certain web sites which contain the wrong domain information and
therefore return 404 errors, viz., ‘file not found’ similar to msn.com links. We use the same
“hack” to find the correct or very similar url and display its data.
(12) Fixed a bug where dropped or copied text containing a Windows wildcard character, e.g., “?” was
considered a bad file name
(13) Fixed a lookup bug where a “%” character in the lookup text was not being encoded correctly
(14) Fixed performance of option to minimize the RCFV window if a browser window is opened from RCFV
(15) Forced url lookup screen to always be ON TOP in case main window is ON TOP
(16) Increased the number of duplicate links within an anchor tag from 1 to 3
– https://nextbigideaclub.com/magazine
(17) Fixed a big which ignored leading spaces when dragging text to the filter input box
(18) Fixed a problem where certain files were being recognized as images even though they weren’t
(19) Fixed a problem with emphasis tags which could create spurious text
(20) Added a new lookup url and associated fvf file to filter only recent Google News
– we have defined “recent” as the last 10 days or so of the previous month plus the current month’s
data but users can redefine it in other ways if they prefer
[Google News — Recent] https://news.google.com/search?hl=en-US&gl=US&ceid=US:en&recentonly&q=??
– check out the new video, “Edit Lookup & Create A Date-Based Filter”, at http://takamomto.com/videos/
which describes how it works
(21) Modified the F2 (lookup) button in the panel such that it is enabled even if there is no file in the viewer
(22) Improved recognition when filtering does not select any meaningful text
(23) Fixed a problem where small text in braces, e.g., {Ctrl}, would be deleted
(24) Speeded up startup and, to a smaller extent, filtering, when there are a large number of filter values
(25) Fixed a potential url history parameter corruption when an invalid numeric value was specified and the
{Enter} key was pressed instead of the RCFV command button
(26) Fixed a problem where a list of image files in the viewer were not recognized as such and, therefore,
were not displayed correctly in the browser when using Copy and edit

plus other minor bug fixes

===============================================================================
Change log for version 1.1
===============================================================================
( 1) Added multipage support for urls with numeric suffixes which precede an extension, e.g., http://www.rickfischer.org/arc001.html
( 2) Added more predefined fractions translated from UNICODE
( 3) Fixed unrecognized left tag bracket in certain urls
( 4) Improved the recognition of images in image tags when the attribute is not obvious, e.g., data-original=”https://images.idgesg.net/images/…”
( 5) Added support for partially “encoded” urls in url pages, e.g., https:u002Fu002Fwww.stitcher.com. This will allow the batch download of Stitcher mp3 files.
( 6) Ensured that changes to a default folder in the Settings will be reflected immediately
( 7) Fixed problem that prevented standard link images from being wrapped horizontally in a browser window
( 8) Fixed Esc key behavior such that it just stops scrolling and does not force a spurious refresh
( 9) Fixed a problem accessing https://www.theskepticsguide.org/podcasts
(10) If the File List is visible but the main window is not then clicking on any line in the list (to go to that file number) will make the main window visible
(11) Some sites, such as www.newsweek.com, hide relevant information and require the “Display hidden text” option
(12) Hopefully improved recognition of web pages which contain non-empty tables. This will result in fewer false warning messages about word-wrap being used with table data.
(13) Fixed next page recognition when it is “hidden” in the ALT attribute text of an image
(14) Allows next page recognition when the assumed next page is significantly different from the initial url but the latter contains a redirect
(15) Repaired several formatting errors when converting web text back to HTML via ‘Copy and edit’
(16) Changed redundant {Alt}+{p} key to {Alt}+{f} for PDF creation in the browser
(17) Using {y} key to toggle filtering caused focus to be lost in viewer
(18) Fixed potential overflow when evaluating images when converting to HTML
(19) The “Display hidden text” option has been extended to include text included within <FORM…> tags
(20) Improved recognition and download of new releases when right-clicking the Help button and selecting option 2
(21) Fixed incorrect handling of some unreadable urls such as https://twitter.com/i/redirect?url=https%3A%2F%2Ftwitter.com….
(22) Added support for image urls without an explicit image extension but which contain the literal string “IMAGE”, e.g., https://www.wsws.org/asset/523f984c-7ad9-403a-bc71-f80e0a82358f?rendition=image1280

plus other miscellaneous, minor problem fixes

---

The following notification is for existing users of RCFV versions prior to version 1.1. New users need not be concerned.

Due to an unfortunate choice of some “default” regular expression meta characters used in the provided fvf files we have discovered that they are too effective at being selected as the “best” matching filter file.

Below is an explanation of the problem if you are interested:

Some of the provided fvf files contained the following regular expression segment (within the quotes), “`S.+`S”. We should have known better than to use the “match everything” wildcard, “.+”, particularly in its greedy mode, but we goofed. Consider the following two pattern files with their equivalent regular expression value and the two urls which follow.

(www`E.)`Qfamousnewspaper.com`S.+`S.fvf
==> (www.)?famousnewspaper.com/.+/
(www`E.)`Qfamousnewspaper.com`S.+`S20`Ed`Ed`S`Ed`Ed`S`Ed`Ed`S.fvf
==> (www.)?famousnewspaper.com/.+/20dd/dd/dd/

https://www.famousnewspaper.com/opinion/
https://www.famousnewspaper.com/opinion/2022/07/01/why_do_so_called_smart_people_do_dumb_things/

One would think that the first pattern would be the best match the first url and that is so. The second pattern should be the best for the second url. Even though both patterns match it, the second regular expression is longer and that is the ostensible tie-breaker. However, we have a problem since RCFV (rightfully) will always choose the first pattern it finds which matches the whole url. Unfortunately, in this case, when you test each pattern for a complete match by adding “^” at the start and “$” at the end (the protocol prefix is always ignored) you get a spurious match for the shorter pattern since the “.+” matches everything after the first slash up to the final slash. 

Here is the fix:

We have replaced all of the provided file names containing the defective string “`S.+`S” with a better pattern, i.e., “`S[^`S]+`S”, which will prevent any unwanted side-effects such as the above from happening. This will happen automatically when you install a new version of RCFV and you allow the installation program to overwrite any matching fvf files.

Again, for new users or those not interested in our predefined fvf files this is not an issue. However, users of older versions will also need to delete the old, imperfect fvf files.

To delete the “bad” files which have been superseded:
(1) Open a command window: type the Windows-key + {R}
(2) Open the CMD (cmd.exe) app
(3) Change the current directory to your filters folder (the one containing your fvf files). For example,
cd “C:Users{your user name}DocumentsRCFVFilters”
(4) Type the following command to delete the “bad” files:
del *`S.+`S*
(5) You can verify that this worked by typing the following command which should return no matches:
dir *`S.+`S*

Sorry for the inconvenience.