Access Chez Scheme documentation from the REPL

In the process of learning Chez Scheme, I missed R’s ability to quickly pull up documentation from the console via help or ?. I was interested in the idea of trying to format the contents of the Chez Scheme User’s Guide (CSUG) and The Scheme Programming Language (TSPL) for display in the REPL (similar to Clojure Docs). That was a bigger task than I wanted to tackle and the first version of chez-docs simply opened links to the relevant sections of CSUG and TSPL from the REPL. Eventually, I more fully scraped the HTML documentation pages and displayed more information in the REPL while still providing the option to open a link to the online documentation.

The initial web scraping was all done in R, but I eventually moved all of that code to Chez Scheme. Because neither CSUG nor TSPL provides a sitemap.xml, sitemaps were generated externally. These sitemaps were parsed using ssax:xml->sxml from the wak library to extract the URL and then the pages were downloaded by shelling out to curl.

The CSUG Summary of Forms page links to both CSUG and TSPL and is used to build a lookup table mapping each procedure name (the key) to its anchor and URL. The HTML is parsed with html->sxml and sxpath is used to extract the table rows. Rows are split by source, and two edge cases are handled manually: alias appears twice in CSUG as both a procedure and a keyword for import. The keyword version is dropped. let appears twice in TSPL pointing to two different anchors. One is renamed "named let".

For each downloaded HTML page, sxpath is used to extract <p> elements and walk through them looking for formdef blocks, the marked-up sections in the documentation that describe each procedure. Everything between a formdef and the next header or footer is collected as the documentation for that anchor. The script handles a variety of HTML entities and inline images (replacing math GIFs with their text equivalents, e.g., =>, ->, lambda), strips HTML attributes, and normalizes whitespace. Tables are replaced with [table not shown]. The result is a nested list keyed by source and anchor.

The library loads the prepared data files directly using include, which requires summary-data.scm and chez-docs-data.scm to be in the same directory as the library file.

The main procedure in chez-docs is doc, which uses case-lambda to handle optional arguments with default values.

(define doc
  (case-lambda
    [(proc) (doc-helper proc 'display 'both)]
    [(proc action) (doc-helper proc action 'both)]
    [(proc action source) (doc-helper proc action source)]))

doc-helper validates arguments, then calls get-doc to look up the documentation text for proc in the given source. get-doc uses get-anchor to map the procedure name to its anchor via summary-data, then looks up the full documentation text in chez-docs-data.

(define (get-doc proc source)
  (let* ([anchor (get-anchor proc source)]
         [src-data (cdr (assoc source chez-docs-data))]
         [doc (assoc anchor src-data)])
    (if doc (cdr doc) doc)))

(define (get-anchor proc source)
  (let* ([src-data (cdr (assoc source summary-data))]
         [row (assoc proc src-data)])
    (if row (cadr row) row)))

When doc is called with 'both, documentation from both sources is displayed when available.

> (doc "<" 'display)

CHEZ SCHEME USER'S GUIDE

(< real1 real2 real3 ...)
...

THE SCHEME PROGRAMMING LANGUAGE

(< real1 real2 real3 ...)
...

action-helper displays the documentation and optionally opens a link to the relevant section in your default browser. When action is 'display, only the text is shown; 'open launches the browser link without displaying text; 'both does both. Opening a link makes a system call to open (macOS), xdg-open (Linux), or start (Windows) and requires an internet connection.

(define (action-helper proc doc action source)
  (let ([srclu '((csug "\nCHEZ SCHEME USER'S GUIDE\n\n")
                 (tspl "\nTHE SCHEME PROGRAMMING LANGUAGE\n\n"))])
    (when (not (symbol=? action 'open))
      (display (cadr (assoc source srclu)))
      (for-each display doc))
    (when (not (symbol=? action 'display))
      (launch-doc-link proc source))))

(machine-type) is used to determine the system-specific string, open-string, for use in the system call.

(define open-string
  (case (machine-type)
    ;; windows
    [(i3nt ti3nt a6nt ta6nt arm64nt tarm64nt)
     "start "]   
    ;; mac
    [(i3osx ti3osx a6osx ta6osx arm64osx tarm64osx ppc32osx tppc32osx)
     "open "]  
    ;; linux
    [(i3le ti3le a6le ta6le arm64le tarm64le arm32le tarm32le
       rv64le trv64le la64le tla64le ppc32le tppc32le)
     "xdg-open "]                      
    [else "undefined"]))

This was a fun little project. When I first had the idea, I was really excited because I worked out all of the initial code in less than 2 hours. But, when I started to write the first version of this blog post, I started to discover all of the little problems that didn’t occur to me initially. Nonetheless, I produced something incredibly useful to myself as an aid for learning and programming in Chez Scheme.