antcolony

8.8 KB

1. # hl:http1 Plugin
3. HTTP/1.1 server plugin for Hybriel. Epoll-based acceptor with I/O thread pool, optional TLS, keep-alive support.
5. ## Two Usage Patterns
7. ### 1. Event-based (class with `on request`)
9. Extend `NativeHttpServer`, override `on request`, call `listen()`.
11. ```hybriel
12. import { NativeHttpServer, Response } from 'hl:http1';
14. MyServer < NativeHttpServer {
15. Number port = 8091;
17. on request(req) {
18. return new Response('<!DOCTYPE>
19. <html>
20. <head>
21. <title>Hello World!</title>
22. </head>
23. <body>
24. <h1>Hello World!</h1>
25. <p>This could be your front page!</p>
26. </body>
27. </html>
28. ', {
29. status = 200;
30. headers = {
31. "Set-Cookie" = 'session=test';
32. };
33. });
34. }
35. }
37. myServer = { port = 8091; } > new MyServer();
38. ```
40. The server runs in the event loop. Each incoming request is dispatched to `on request` as a fiber, so the process stays alive and can handle concurrent requests.
42. ### 2. Iterator-based (`for (req of server)`)
44. Call `listen(port)` directly, iterate requests in a loop.
46. ```hybriel
47. import { listen, Response } from 'hl:http1';
49. server = listen(9123);
51. for (req of server) {
52. if (req.path == "/hello") {
53. emit req.response(new Response('<!DOCTYPE>
54. <html>
55. <head>
56. <title>Hello World!</title>
57. </head>
58. <body>
59. <h1>Hello World!</h1>
60. <p>This could be your front page!</p>
61. </body>
62. </html>
63. ', {
64. status = 200;
65. headers = {
66. "Set-Cookie" = 'session=test';
67. };
68. }));
69. } else {
70. emit req.response(new Response('Not found', { status = 404; }));
71. }
72. }
73. ```
75. This blocks on each iteration waiting for the next request. Simpler but sequential.
77. ## Request Object
79. Each request has these fields:
81. | Field | Type | Description |
82. |-------|------|-------------|
83. | `method` | String | `"GET"`, `"POST"`, etc. |
84. | `path` | String | `"/hello"`, `"/api/users"` |
85. | `query` | Object | Parsed query params (keys/values) |
86. | `headers` | Object | HTTP headers (keys/values) |
87. | `body` | String or null | Request body — a `Content-Length` one, or a `Transfer-Encoding: chunked` one de-chunked |
88. | `bytes` | Bytes | The same body as a Bytes, byte for byte (empty when there is none) |
89. | `respond` | Handle | Response handle |
91. ## Response Object
93. Create a `Response` with body and optional options:
95. ```hybriel
96. // Basic
97. new Response("Hello!")
99. // With status and headers
100. new Response('{"ok":true}', {
101. status = 200;
102. headers = {
103. "Content-Type" = 'application/json';
104. "Set-Cookie" = 'session=abc';
105. };
106. })
108. // Error response
109. new Response('Not found', { status = 404; })
110. ```
112. | Field | Type | Default | Description |
113. |-------|------|---------|-------------|
114. | `body` | String or Bytes | `''` | Response body; a Bytes is sent raw, as `application/octet-stream` unless `headers` name a type |
115. | `status` | Number | `200` | HTTP status code |
116. | `headers` | Object | `{}` | HTTP headers (key-value pairs) |
118. ## NativeHttpServer Properties
120. | Property | Type | Default | Description |
121. |----------|------|---------|-------------|
122. | `port` | Number | 8080 | Listen port |
123. | `host` | String | "0.0.0.0" | Bind address |
124. | `tls_cert` | String | null | Path to TLS certificate |
125. | `tls_key` | String | null | Path to TLS private key |
126. | `threads` | Number | 4 | I/O worker threads |
128. ## Architecture
130. ```
131. Client → epoll acceptor thread → PARK (epoll) → I/O thread pool (TLS + parse)
132. → request queue → event loop → on request fiber
133. ```
135. - Acceptor thread uses epoll to accept connections without blocking
136. - I/O workers handle TLS handshake and HTTP parsing in parallel
137. - Parsed requests go into an MPSC queue
138. - The runtime event loop polls the queue (non-blocking) and spawns a fiber per request
140. ### Parking: an idle connection costs no thread (mission 084)
142. A worker's `read()` is blocking, so a connection handed to the pool occupies a whole
143. worker until bytes arrive. Keep-alive connections used to be re-enqueued straight into the
144. pool after each response, so an idle one still held a thread — with the default
145. `threads = 4`, **four quiet sockets starved the server** and every later request hung
146. (mission 075 saw this as "the second browser session's page never fires `load`").
148. Connections are therefore **parked** in a dedicated epoll instance (`ParkedConns`) and only
149. enter the worker queue once they are actually readable — or have hung up, which a worker
150. reads as EOF and closes. Both the acceptor and the post-response keep-alive path park.
151. An idle connection now costs one fd and zero threads.
153. - A parked connection that stays silent for `PARK_IDLE_TIMEOUT_MS` (60s) is closed.
154. - Workers set `SO_RCVTIMEO` (30s) on every connection as a backstop against a client that
155. trickles headers; it is no longer the idle-timeout mechanism.
156. - **An established TLS connection is never parked**: OpenSSL may hold already-decrypted
157. plaintext that epoll on the raw fd cannot see. Those go straight to a worker, as before,
158. so a TLS server is still starvable by idle keep-alive connections — the plain-HTTP path
159. (and the pre-handshake TLS socket, whose ClientHello does arrive on the raw fd) is fixed.
160. - Regression test: `tests/browser/tests/13-keepalive-starvation.mjs`.
162. ### WebSocket liveness sweep (mission 091)
164. The WS reader loop (one thread, epoll, level-triggered EPOLLIN) already wakes on a 500ms
165. timeout, so liveness costs no timer and no extra thread: on every tick it walks the
166. upgraded sockets, pings the ones that have gone quiet, and drops the ones whose pong is
167. overdue.
169. A drop enqueues the **same `close` event a real FIN would have produced**, so every
170. consumer above this plugin — hl:web's push subscription registry included — prunes through
171. the path it already had. No new concept travels upward, and nothing at the `.hl` level
172. needs a timer facility.
174. Why it is needed: a peer that vanishes without closing (dead wifi, a suspended laptop, a
175. host that never sent the FIN) leaves a socket that stays open and **writable**. A server
176. with nothing to push at it never discovers the corpse, so before this the subscription was
177. immortal. A closed tab is not this case — the browser's stack sends the FIN, and even under
178. CDP offline emulation it still answers protocol pings (measured, mission 091), which is why
179. the regression test's dead peer is a raw socket that goes silent rather than a tab.
181. | Env var | Default | Meaning |
182. |---------|---------|---------|
183. | `HL_WS_PING_MS` | `15000` | a socket quiet this long is pinged |
184. | `HL_WS_PONG_TIMEOUT_MS` | `10000` | a ping unanswered this long ⇒ the peer is gone |
186. Read once when the engine starts; timeouts are measured on `CLOCK_MONOTONIC`, so an NTP
187. step cannot make one fire early or never. Any inbound frame — of any opcode — counts as
188. proof of life and clears an outstanding ping.
190. - Regression test: `tests/browser/tests/23-ws-liveness-sweep.mjs` (includes the
191. falsification: the same silent peer against a sweep-disabled server stays registered).
193. ### The handshake's cookies + a cookie-grade random (mission 093)
195. A WebSocket upgrade is an ordinary HTTP request, so the browser attaches the site's cookies
196. to it. That is the one moment an upgraded socket can be attributed to whoever loaded the
197. page — after the 101 the request and its headers are freed and the connection carries no
198. identity of its own. The `connect` event therefore gained a sixth field:
200. ```
201. { kind, id, data, binary, code, cookie }
202. ```
204. `cookie` is the handshake's `Cookie:` header verbatim, non-empty on `connect` only. Nothing
205. is parsed here: hl:web's session layer (`plugins/web/web_session.hl`) owns the cookie's name
206. and meaning, and this plugin owns only its delivery.
208. ```
209. randomToken(Number n) // n lowercase hex chars, kernel CSPRNG (getrandom(2)), max 128
210. ```
212. A session id is the only thing between a stranger and someone else's session, so it may not
213. come from a seeded PRNG — `hl:math`'s `random()` is a clock-seeded xoshiro whose state a
214. handful of outputs reveals, which would make every other id derivable from one's own. This
215. lives here because a session cookie is an HTTP artifact and this is the plugin that parses
216. and writes one; it is currently the only CSPRNG reachable from `.hl`.
218. - Regression test: `tests/browser/tests/25-sessions.mjs` (two real browser instances = two
219. cookie jars = two sessions).
221. Ticket #74 added `host` (the handshake's `Host:` header) and ticket #105 `headers`: every
222. header of the handshake as `name: value` lines, names lowercase, non-empty on `connect` only.
223. `NativeWebSocketServer` hands them to the `WsClient` as `client.headers`, a hash by name —
224. which is where hl:web reads the headers a navigation over that socket is constructed with.