How do I use the Debug Console for a website project?

I’m looking to run a web app debugger in the IDE similar to how you can in VS Code.
The documentation implies that this is possible:

Project IDX includes the built-in Debug Console from Code OSS. Use this console to debug your app …

But when I run the debugger I get the message “All debug extensions are disabled. Enable a debug extension or install a new one from the Marketplace.”

There don’t seem to be any good options in the marketplace.

I’ve tried this with the Astro, and Simple HTML templates. Same result.