Rethinking the Web – Part 1

What would the web look like if the pioneers had the tools and tech we have today? What would the OO software look like if we used the OO paradigm championed by the developers of the web? Why has software development become so complicated? This series of articles answers these questions, and presents an alternate approach to how we design and develop software.

Preface

In the mid 1990’s software development was reasonably predictable. The Client-Server model was well established, methodologies were solid, and applications were fast and (in an industrial way) pretty. Like the peak of the Roman Empire, things were good. Then, the web swept in like the barbarian hordes, with it’s primitive scripts and html. While users wept about performance and quality (just Wayback Machine to web “apps” in the early 90’s), management did nothing. Why?

The initial explosion of the web in the enterprise was really based on a single factor. Distribution. At the time, every update to every application required a trip to the users desk with a CD.

As the web was taking form and approaches being solidified, enterprises were consumed with Y2K Remediation. Everyone else was trying to get on the web, be found there, and keeping users from clicking away. Tools and methodologies supported this, and the technology core of the web was built around CSS and SEO.

Things have changed. SEO is mostly pointless now. Crawlers are going to index you pretty well anyway, but search results are now Pay to Play for your site to show up. CSS has become pervasive and invasive. It could be the Jeopardy response to, “It laughs at the Single Responsibility Principle”.

It’s taken over so much there are people out there that actually think the `class` attribute on an element means CSS Class!

In spite of all of this (and more) implementors and developers are doing some awesome things. The Document based approach works well for Informational, Promotional, and Catalog type sites – they have paper document analogs. But the model isn’t getting enterprise apps back to their former glory.

Add to this our Object Model. During the 60’s and 70’s two general approaches to Object Orientation emerged. One thought if objects as “things”, the other thought of objects as “beings”. The mainstream way we see object today is neither. It’s is a variation of the “things” approach, where the things are like RDBMS tables. The Association in  OO Theory is implemented as a DB style relationship. 

And like many things the software is packed with dogma and dogma spouting parrots. Things like DRY, SOLID, and a host of others are just accepted. Are they still valid in the 20 years later world? 

I started a “back to basics” project, with a focus on the needs of the enterprise. I took the tools, techs, and knowledge we have today back to the 60’s and started over. 

These articles share what I’ve learned and created.   

Introduction

This article provides an overview of WebSockets and shows how to remote control browsers over sockets. The example uses a Windows Console to send and execute code on the browser.  

The Parts

Browsers work in a Request-Response mode. But they can also work in a Solicit-Response mode. A basic difference between these is direction. Clients initiate Requests, Servers initiate Solicits. Another difference is control. In an RR application the browser/client controls the workflow. In an SR application that is inverted – the server is in control. Using Solicit-Response we can remote control our browsers.

HTTP doesn’t support solicits, but WebSockets can.

WebSockets

Creating a socket is easy: (script)

new WebSocket(location.origin.replace("http", "ws"));

Sockets are same-origin, different protocol, so we can use a little replace hack.
This sends an HTTP GET with some socket related info in the header .

On the server side it’s just as easy: (C# concept code – don’t.) 

MapGet("/", async (HttpContext http) => await http.WebSockets.AcceptWebSocketAsync( ));

The server (in C#) creates an object (HttpContext) with a WebSockets interface.
Note: I will be using interface in the common way. If I mean a C# Type (IWhatever) I will specify.
The Accept method sends a transparent response to the browser that triggers the onopen event of the socket.

An open socket can’t really do much. It needs to be wired into the system. If the server started sending messages right away the would essentially go nowhere. No one is listening. We need listeners. And, we need to know we have listeners.

self.socket = new WebSocket(location.origin.replace("http", "ws"));
socket.onopen = () => 
  socket.onmessage = () => ...);
    socket.send("ready");
  

In the onopen handler we can attach an onmessage handler, and do any other initialization work. When finished, we let the server know all is well – “ready”.

Sockets on the server are a little different than we might expect from C# objects. While many messaging components “push” (e.g., Events/Delegates). We need to explicitly wait on sockets.
>Note: A current issue with sockets is you can’t stop waiting. Canceling kills the connection.

WebSocket socket = await http.WebSockets.AcceptWebSocketAsync();
var buffer = new byte[1024];
WebSocketReceiveResult ack = await socket.ReceiveAsync(buffer, CancellationToken.None);

The “ack” object is a status object. The data/message is on the buffer, and easy to get as text. 

string msg = Encoding.UTF8.GetString(buffer[...ack.Count]);

There is one thing left to do. Sockets are like nested connections – a socket connection inside an HTTP connection. If we don’t do anything the connections will fall though or time out. The infrastructure handles HTTP with built in “keep-alive”. The HTTP part is basically unused we tell it to wait for a response and never send one. The “keep-alive” extends the lie (over and over) – “Just 30 sec more…” (Cruel).

While the HTTP connection is open a WebSocket connection can live inside. By default and socket will close on its own after sending a message. We need to keep it open too. (e.g., By looping over the receive waiter).

That’s really about 90% of WebSockets. The rest is variations of this (i.e., binary messages) or plumbing.

Browser Tasks

The browser “JavaScript Virtual Machine” (JVM) that runs JavScript is an interesting beast. While it has been improved and augmented, the basic architecture hasn’t changed since V1. Internally it appears much like a Motorola 6800/68000 CPU from the 1970/80’s. Programming takes me back to BASIC (not in syntax, but approach) and batch files in DOS or Mainframe schedulers. Not bashing. Those are all good tech’s, but slightly outdated in this age of distributed asynchronous computing. This topic is explored in other articles.

What is relevant now is that browser code is interpreted. Ignoring any optimizations, an Interpreter knows nothing about interpreted code until it sees it. It doesn’t matter when or how code gets there, as long as it is there when needed. A perfect environment for a Just In Time approach.

Script supports Functions (function() ), which can be asynchronous. The async function constructor isn’t surfaced on it’s own. This code will build and run an async or “standard” function:

self.AsyncFunction = Object.getPrototypeOf(async function ()  ).constructor;
self.Execute = async (cmd) => await (new AsyncFunction(cmd))() ?? "Success"

This runs any valid JavaScript code. If the code return a value Execute returns that value. “Success” is returned if it completes with no errors and no return value.

Plugged in the socket code:

self.AsyncFunction = Object.getPrototypeOf(async function ()  ).constructor; 
self.Execute = async (cmd) => await (new AsyncFunction(cmd))() ?? "Success"

self.socket = new WebSocket(location.origin.replace("http", "ws"));
socket.onopen = async () => 
    socket.send("ack");
    socket.onmessage = async (msg) => 
        try  socket.send(await Execute(msg.data)); 
        catch (e)  socket.send(`Fail: $e`); 
        
    ;

This builds a function from a message “Just in Time”, executes it, and sends the result back through the socket. It operates (ignoring optimizations) exactly like a function created on page load.

The server side uses the send and wait lines: 

string cmd = "return 4+5;"
var buffer = new byte[1024];
_ = socket.SendAsync(new(UTF8.GetBytes(cmd)), Text, true, _ct.None);
var result = await socket.ReceiveAsync(buffer, _ct.None);
string data = Encoding.UTF8.GetString(buffer[..result.Count]);

Variable data will equal “9”. Leaving off the return (“4+5”) will put “Success” in data.

With this approach we can send tasks (in Script) to the browser from the server.

Server

To support this stuff we need a web server. The relevant server code right now is:

_ = app.MapGet("/", (HttpContext http) => http.WebSockets.IsWebSocketRequest switch {
    false => http.Response.WriteAsync("<!DOCTYPE html><html>%SocketCode%</html>"),
    true => Task.Run(async () => {
        WebSocket socket = await http.WebSockets.AcceptWebSocketAsync();
           ....

HTTP and WebSocket requests can map to the same url. The socket interface on HttpContext provides an IsWebSocketRequest property to differentiate. We can simply switch on this.
When true we start the socket process. When false a standard HTML Document. Or something else.

Browsers are pretty tolerant. If we don’t provide a doc it will create one. All we care about from the HTTP request is that we get the socket code run. We can put the document in with:

 self.SetDocument = (content) =>  
      const doc = document.open(); doc.write(content); doc.close();  

We can just send (made easy with raw string literals):

http.Response.WriteAsync($$"""
  <script>
    self.AsyncFunction = Object.getPrototypeOf(async function ()  ).constructor;
    self.Execute = async (cmd) => await (new AsyncFunction(cmd))() ?? "Success"
    self.SetDocument = (content) =>  const doc = document.open(); 
         doc.write(content); doc.close(); 
    self.SetSheet = (content) =>  const s = new CSSStyleSheet(); s.replace(content);
         document.adoptedStyleSheets = [s]; 
    self.socket = new WebSocket(location.origin.replace("http", "ws"));
    socket.onopen = async () => 
      socket.send("ack");
      socket.onmessage = async (msg) => 
        try  socket.send(await Execute(msg.data)); 
        catch (e)  socket.send(`Fail: $e`); 
        
    ;
  </script>
  """);

Replacing the document through code isn’t like an HTTP GET. It doesn’t reset the window or execution context. It only replaces what is between the html tags. Everything we mount in the EC self.Thing = …  stays, and is available for the new document.

Not only can we replace documents and parts dynamically, we can do this with CSS. Notice the SetSheet function. We can take “full control” of a browser putting in or removing and CSS, Markup, or Script in realtime.

Important Concept Point

With this approach we can control the browser execution context and content in real time. 

There never needs to be “just in case” code, markup, or pages of CSS. The only code that needs to be in the browser is the currently visible artifacts. Code can be “fabricated” (another article) in real-time to fit specific conditions and states.

All Together Now…

Here is a working application. It uses a Windows Console to control a browser.
If your IDE starts a browser turn that off. The code is .net7 + raw string literals.

In VS, Project File and Launch:

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net7.0</TargetFramework>
    <Nullable>enable</Nullable>
    <LangVersion>preview</LangVersion>
  </PropertyGroup>
</Project>
  "profiles": 
    "http": 
      "commandName": "Project",
      "launchBrowser": false,
      "applicationUrl": "http://localhost:5150"
    
  

Code:

using System.Threading;
namespace CP;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using System;
using System.Diagnostics;
using System.Net.WebSockets;
using System.Threading.Tasks;
using static System.Net.WebSockets.WebSocketMessageType;
using static System.Text.Encoding;
using _ct = CancellationToken;

public class Program {
    public static void Main(string[] args) {

        int port = 5085;
        Barrier barrier = new(2);
        WebApplicationBuilder builder = WebApplication.CreateBuilder();
        builder.WebHost.PreferHostingUrls(true).ConfigureKestrel(s => s.ListenLocalhost(port));
        WebApplication app = (WebApplication)builder.Build().UseWebSockets();

        _ = app.MapGet("/", (HttpContext http) => http.WebSockets.IsWebSocketRequest switch {

            false => http.Response.WriteAsync($$"""
              <script>
                self.AsyncFunction = Object.getPrototypeOf(async function ()  ).constructor;
                self.Execute = async (cmd) => await (new AsyncFunction(cmd))() ?? "Success"
                self.SetDocument = (content) =>  const doc = document.open();
                  doc.write(content); doc.close(); 
                self.SetSheet = (content) =>  const s = new CSSStyleSheet(); s.replace(content);
                  document.adoptedStyleSheets = [s]; 
                self.Read = (facet, field) => document.querySelector(`[facet="$facet"][field="$field"]`).value;
                self.Write = (facet, field, v) => document.querySelector(`[facet="$facet"][field="$field"]`).value = v;
                self.socket = new WebSocket(location.origin.replace("http", "ws"));
                socket.onopen = async () => 
                  socket.send("ack");
                  socket.onmessage = async (msg) => 
                    try  socket.send(await Execute(msg.data)); 
                    catch (e)  socket.send(`Fail: $e`); 
                    
                  ;

                self.NicheNode = class NicheNode extends HTMLElement  #content = null; #name = null;
                  static Replace(niche, content)  
                    const n = document.querySelector(`layout-niche[niche="$niche"]`)
                    .replaceWith(new NicheNode(niche, content)); 

                  constructor(name, content)  super(); 
                     if (content) this.#content = content; if (name) this.#name = name; 

                  connectedCallback() 
                    if (this.#content != null) this.innerHTML = this.#content;
                    if (this.#name != null) this.setAttribute("niche", this.#name);
                    
                  ;
                customElements.define("layout-niche", NicheNode);

              </script>
              """),

            true => Task.Run(async () => 
                WebSocket socket = await http.WebSockets.AcceptWebSocketAsync();
                var buffer = new byte[1024];
                WebSocketReceiveResult ack = await socket.ReceiveAsync(buffer, _ct.None);
                barrier.SignalAndWait();

                _ = socket.SendAsync(new(UTF8.GetBytes($$"""
                  SetDocument(`
                  <!DOCTYPE html><html lang="en">
                    <head><title>Wisp</title>
                    </head>
                    <body>
                      <area-left>
                        <layout-niche niche="left"></layout-niche>
                      </area-left>
                      <area-right>
                        <layout-niche niche="left">
                          <input type="text" facet="person" field="name" value="" />
                        </layout-niche>
                      </area-right>
                    </body>
                  </html>
                  `);
                  """)), Text, true, _ct.None);
                _ = await socket.ReceiveAsync(buffer, _ct.None);

                _ = socket.SendAsync(new(UTF8.GetBytes($$"""
                  SetSheet(`
                  body  display:flex; gap:10px; 
                  area-left, area-right  display:flex; flex-direction:column; 
                      border: 1px solid black; min-height: 200px; 
                  area-left  flex:1; 
                  area-right  flex:2; 
                  `);
                  """)), Text, true, _ct.None);
                _ = await socket.ReceiveAsync(buffer, _ct.None);

            Loop:
                string cmd = "";
                Console.Write(">");
                if( (cmd = Console.ReadLine() ?? throw new()) == "" ) goto Loop;
                await socket.SendAsync(new(UTF8.GetBytes(cmd)), Text, true, _ct.None);
                var status = await socket.ReceiveAsync(buffer, _ct.None);
                Console.WriteLine(UTF8.GetString(buffer[..status.Count]));
                goto Loop;
            )
            });

        Task.Factory.StartNew(async () =>  await app.StartAsync(); );
        _ = Process.Start("explorer", $"http://localhost:port");
        barrier.SignalAndWait();
        _ = new AutoResetEvent(false).WaitOne();
        }
    }

You should see a “>” prompt in the Windows Console.

Try these commands:

1.a return 4+5;
1.b 4+5

2. NicheNode.Replace(“left”, `<h3>Replacement</h3>`);

3. SetSheet(`body background-color: blue;  `);

This blows away the old styling. In a future article I cover Sheet Management.
Just restore it.

4. SetSheet(` body display:flex; gap:10px; area-left, area-right display:flex; flex-direction:column; border: 1px solid black; min-height: 200px; area-left flex:1; area-right flex:2; `);

Solicit

In a Request-Response model we wait for a user to post/submit and they send some predefined data. In a Solict-Response model we just go get what we want, when we want.

5. Write(“person”, “name”, “Gwyll”);

6 Read(“person”, “name”);

Closing

In the next article I will expand on this foundation.
I hope you found this interesting.

History


This member has not yet provided a Biography. Assume it’s interesting and varied, and probably something to do with programming.