Reading and writing JSON files in R and Racket

In learning about reading CSV files in Racket, I have started to reconsider whether storing small(ish) datasets in CSV files is the best default behavior.1 My default was set by primarily working in R, where reading and writing CSV files plays a central role in data analysis. When working solely in R, I expect that my old habits will die hard and CSV files will continue to play a prominent role. However, when passing small(ish) data between R and Racket, I think JSON might be a better alternative.

Before we jump into a bunch of code examples, let’s define some wrapper functions to make the subsequent code a bit more concise. In R, we are using the jsonlite package and a little function to write the JSON object to file.

library(jsonlite)

write_json <- function(object, filename){
  writeLines(toJSON(object), filename)
}

In Racket, we are using the json module and have simple wrapper functions for both reading and writing JSON. A jsexpr is a subset of Racket values (e.g., list, hash table, etc.) that can be represented as JSON strings.

#lang racket

(require json)

(define (read-json-wrapper filename)
  (call-with-input-file filename read-json))

(define (write-json-wrapper jsexpr filename)
  (call-with-output-file filename
    (λ (x) (write-json jsexpr x))
    #:exists 'replace))

R Data Structures

Data Frame

The data frame is the central data structure in R, which corresponds well to flat, delimited text files (in contrast to JSON). In this code chunk, we use the same example CSV file as in the CSV reading post.

> write_json(head(read.csv("example.csv")), "R_dataframe.json")
> fromJSON("R_dataframe.json")
        date integer         float  bool char      word yn
1 12/25/2060 -958838  590131109036  true    u      zosu  Y
2 02/18/2023 -165090  528052918839  true    1        em  Y
3 11/27/2064 -207296   99397538939 false    w    doptah  Y
4 06/28/2044  316236  590216172038 false    x gutijenel  N
5 08/12/2045  722897 -309360363517  true    7      hueh  N
6 07/23/1904 -903116 -509332808532  true    0   fufcora  Y

Racket reads the data frame as a list of hash tables. Note that keys (e.g., bool, char, date, etc.) in the hash table are listed in alphabetical order. Also, the redundancy of the JSON format is evident for this tabular data via the repetition of the column names.

> (read-json-wrapper "R_dataframe.json")
'(#hasheq((bool . "true")
          (char . "u")
          (date . "12/25/2060")
          (float . 590131109036.032)
          (integer . -958838)
          (word . "zosu")
          (yn . "Y"))
  #hasheq((bool . "true")
          (char . "1")
          (date . "02/18/2023")
          (float . 528052918838.886)
          (integer . -165090)
          (word . "em")
          (yn . "Y"))
  #hasheq((bool . "false")
          (char . "w")
          (date . "11/27/2064")
          (float . 99397538938.88)
          (integer . -207296)
          (word . "doptah")
          (yn . "Y"))
  #hasheq((bool . "false")
          (char . "x")
          (date . "06/28/2044")
          (float . 590216172037.734)
          (integer . 316236)
          (word . "gutijenel")
          (yn . "N"))
  #hasheq((bool . "true")
          (char . "7")
          (date . "08/12/2045")
          (float . -309360363516.723)
          (integer . 722897)
          (word . "hueh")
          (yn . "N"))
  #hasheq((bool . "true")
          (char . "0")
          (date . "07/23/1904")
          (float . -509332808531.968)
          (integer . -903116)
          (word . "fufcora")
          (yn . "Y")))

Vector

It is unlikely that I would want to write a vector to file, but it is straightforward with jsonlite.

> write_json(vector(mode = "numeric", length = 10), "R_vector.json")
> fromJSON("R_vector.json")
 [1] 0 0 0 0 0 0 0 0 0 0

Racket reads the vector as a list.

> (read-json-wrapper "R_vector.json")
'(0 0 0 0 0 0 0 0 0 0)

Matrix/Array

The jsonlite package handles both matrices and arrays.

> write_json(matrix(data = runif(25), nrow = 5, ncol = 5), "R_matrix.json")
> fromJSON("R_matrix.json")
       [,1]   [,2]   [,3]   [,4]   [,5]
[1,] 0.1451 0.3178 0.3240 0.9761 0.2404
[2,] 0.9971 0.1716 0.3767 0.1735 0.7416
[3,] 0.5081 0.7600 0.6993 0.2717 0.1228
[4,] 0.2520 0.5952 0.1561 0.4297 0.4912
[5,] 0.6209 0.8862 0.4749 0.4307 0.2534

Racket reads a matrix as a list of lists. It can also read multidimensional arrays by further nesting of lists.

> (read-json-wrapper "R_matrix.json")
'((0.1451 0.3178 0.324 0.9761 0.2404)
  (0.9971 0.1716 0.3767 0.1735 0.7416)
  (0.5081 0.76 0.6993 0.2717 0.1228)
  (0.252 0.5952 0.1561 0.4297 0.4912)
  (0.6209 0.8862 0.4749 0.4307 0.2534))

List

JSON is a hierarchical data format that is well suited for nested lists.

> write_json(list("A" = list("One" = 1, 
+                            "Two" = 2,
+                            "Three" = 3),
+                 "B" = list("Four" = 4, 
+                            "Five" = 5,
+                            "Six" = 6)),
+            "R_list.json")
> fromJSON("R_list.json")
$A
$A$One
[1] 1

$A$Two
[1] 2

$A$Three
[1] 3


$B
$B$Four
[1] 4

$B$Five
[1] 5

$B$Six
[1] 6

Racket reads the nested list as a nested hash table2 rather than a list of hash tables as with the data frame. As with reading the data frame above, the keys are listed in alphabetical order.

> (read-json-wrapper "R_list.json")
'#hasheq((A . #hasheq((One . (1)) (Three . (3)) (Two . (2))))
         (B . #hasheq((Five . (5)) (Four . (4)) (Six . (6)))))

Racket Data Structures

List

In this first example, I’ve just thrown together a bunch of the different Racket values that are valid as jsexpr.

> (write-json-wrapper (list 1 2.1 #t "B" "word" (list 3 4) (hash 'C 5 'D '(6 -7))) "Racket_list.json")
> (read-json-wrapper "Racket_list.json")
'(1 2.1 #t "B" "word" (3 4) #hasheq((C . 5) (D . (6 -7))))

The jsonlite package reads the Racket list as an R list. Note that the boolean value is automatically converted to a logical value in R (TRUE). Also, the keys (C, D) in the hash table are used as names in the nested list.

> fromJSON("Racket_list.json")
[[1]]
[1] 1

[[2]]
[1] 2.1

[[3]]
[1] TRUE

[[4]]
[1] "B"

[[5]]
[1] "word"

[[6]]
[1] 3 4

[[7]]
[[7]]$D
[1]  6 -7

[[7]]$C
[1] 5

Hash Table

If you’ve read this far, there should be no surprise about how suitable JSON is for a nested hash table.

> (write-json-wrapper (hasheq 'A (hasheq 'C 2 'E 6) 'B (hasheq 'D 4 'F 8)) "Racket_hash.json")
> (read-json-wrapper "Racket_hash.json")
'#hasheq((A . #hasheq((C . 2) (E . 6))) (B . #hasheq((D . 4) (F . 8))))

The nested hash table becomes a nested (and named) list in R.

> fromJSON("Racket_hash.json")
$A
$A$E
[1] 6

$A$C
[1] 2


$B
$B$D
[1] 4

$B$F
[1] 8

Table

This table is actually a list of lists where the first row of the list contains the headers for the columns.

> (write-json-wrapper (list '("A" "B" "C") '(1 2 3) '(4 5 6)) "Racket_table.json")
> (read-json-wrapper "Racket_table.json")
'(("A" "B" "C") (1 2 3) (4 5 6))

jsonlite reads this “table” as a matrix and converts all elements to characters because all elements of an R matrix need to be the same type.

> fromJSON("Racket_table.json")
     [,1] [,2] [,3]
[1,] "A"  "B"  "C" 
[2,] "1"  "2"  "3" 
[3,] "4"  "5"  "6" 

List of Hash Tables

A list of hash tables was how the json module read the JSON file created by jsonlite for a data frame.

> (write-json-wrapper (read-json-wrapper "R_dataframe.json") "Racket_list_hash.json")
> (read-json-wrapper "Racket_list_hash.json")
'(#hasheq((bool . "true")
          (char . "u")
          (date . "12/25/2060")
          (float . 590131109036.032)
          (integer . -958838)
          (word . "zosu")
          (yn . "Y"))
  #hasheq((bool . "true")
          (char . "1")
          (date . "02/18/2023")
          (float . 528052918838.886)
          (integer . -165090)
          (word . "em")
          (yn . "Y"))
  #hasheq((bool . "false")
          (char . "w")
          (date . "11/27/2064")
          (float . 99397538938.88)
          (integer . -207296)
          (word . "doptah")
          (yn . "Y"))
  #hasheq((bool . "false")
          (char . "x")
          (date . "06/28/2044")
          (float . 590216172037.734)
          (integer . 316236)
          (word . "gutijenel")
          (yn . "N"))
  #hasheq((bool . "true")
          (char . "7")
          (date . "08/12/2045")
          (float . -309360363516.723)
          (integer . 722897)
          (word . "hueh")
          (yn . "N"))
  #hasheq((bool . "true")
          (char . "0")
          (date . "07/23/1904")
          (float . -509332808531.968)
          (integer . -903116)
          (word . "fufcora")
          (yn . "Y")))

Apparently, the json module keeps track of the original order of the keys (even though they are sorted in the hash table) because the order of the columns in the data frame is preserved.

> fromJSON("Racket_list_hash.json")
        date  bool integer      word yn char         float
1 12/25/2060  true -958838      zosu  Y    u  590131109036
2 02/18/2023  true -165090        em  Y    1  528052918839
3 11/27/2064 false -207296    doptah  Y    w   99397538939
4 06/28/2044 false  316236 gutijenel  N    x  590216172038
5 08/12/2045  true  722897      hueh  N    7 -309360363517
6 07/23/1904  true -903116   fufcora  Y    0 -509332808532

Conclusion

My primary use case for JSON files would be to create lookup tables in R for input values that are used in simulation models built in Racket. In R, I have tended to use data frames for lookup tables because a data frame is the typical output of a data analysis pipeline. I will need to learn more about working with lists of hash tables and nested hash tables in Racket to decide if it makes sense to reformat a data frame into a nested list before writing the object to JSON.


  1. I even used CSV files as a database replacement in my Shiny Scorekeeper app.↩︎

  2. See also previous post on the similarity of nested lists in R and nested hash tables in Racket.↩︎

Avatar
Travis Hinkelman
Ecological Modeler

Related