Blog Articles and information on C# and .NET development topics

Note that WebCopy 1.10 and Sitemap Creator 1.3 onwards no longer supports Windows Vista, 8 and earlier versions of Windows 10.

WebCopy 1.7 introduced website authentication using Internet Explorer. Even at the time I knew it was a stop-gap solution and long term the plan was to have WebCopy support both Gecko and Chromium engines for advanced support. Of course, in Cyotek-land plans never come to fruition.

In the interim period Microsoft introduced the new Chromium based Edge and then Edge WebView2. WebCopy users can now choose to use either Internet Explorer or a system installation of WebView2 for advanced GUI based tasks.

To clarify, you cannot (yet) use it to scan websites to work around WebCopy's inability to execute JavaScript whilst crawling. Its main use case is for manually authenticating with a website to allow WebCopy to borrow the cookies.

Although you almost certainly have an uninstallable version of Edge installed that doesn't necessarily mean you have WebView2 installed - see the Microsoft download page for installation details. Note that WebCopy currently doesn't support fixed versions, but only the evergreen system component.

The Options dialogue now includes a new Embedded Web Browser category which allows you to choose between IE and WebView2.

One of the interesting things about WebView2 is it doesn't share global state with the Edge browser itself. A new option has been provided allowing you to configure the location where the WebView2 profile is stored, along with an option to clean it up after each use (e.g. for deleting cookies and clearing cached files).

The dialogue will also check to see if WebView2 is installed before allowing it to be selected.

This should still be considered experimental functionality. And by using WebView2, Microsoft probably siphons off half your hard disk to their servers as per their egregious data collection practices.

What is also quite not-funny is one of the reasons users requested this functionality is to log into Google websites. But I understand that Google in their position of "Our Way Or The Highway" actively block embedded browsers such as WebView2 for being used for authentication. But ignoring that, at least it can be used for modern websites where IE now falters.

The default option remains as Internet Explorer given it is already present on users machines and WebView2 may not be.

Finally, although technically WebCopy can still be used on Windows Vista, WebView2 is only available on Windows 7, Windows 8.1, later versions of Windows 10 and any version of Windows 11 (and even then the Windows 7 and 8.1 versions are no longer supported).

The title of this post is part 1 - suggesting a part 2. And this is the part 2 to follow!

WebCopy is a relatively old piece of software - the first version was released June 15th, 2010. In addition, it deliberately used old versions of the .NET Framework to allow it to be used on older versions of Windows. These restrictions have gradually been removed, but the underlying code is still using old coding methods. So while modern code (like WebView2) uses the "async and await" patterns, WebCopy uses older constructs like AsyncOperationManager and IAsyncResult. A minimum of refactoring has been done to inject WebView2 for use in GUI dialogues like External Login and Capture Form, but changing the crawl engine to be able to make use of this is a bigger task. So for now, it can be used for manual authentication, and in a future update you will (should) be able use it to complement WebCopy's crawl engine if desired.

While we appreciate comments from our users, please follow our posting guidelines. Have you tried the Cyotek Forums for support from Cyotek and the community?