plz-media-type

Next:   [Contents]

plz-media-type

plz-media-type is an Emacs Lisp library that enhances MIME type handling for HTTP requests within Emacs. It leverages the plz.el HTTP library for networking calls and introduces a mechanism to process responses based on the content type header. This library defines various classes and methods for parsing and processing standard MIME types, including JSON, XML, HTML, and binary data. It is used in the LLM library to handle decoding of various streaming and non-streaming media type formats.

Table of Contents


Next: , Previous: , Up: plz-media-type   [Contents]

1 Installation


1.1 GNU ELPA

plz-media-type is available in GNU ELPA. It may be installed in Emacs using the package-install command.


2 Usage

The main function, plz-media-type-request, works similarly to the plz function but with an additional feature. When the :as option is set to (media-types MEDIA-TYPES), the HTTP response is decoded based on the content type header. The MEDIA-TYPES association list defines a mapping from content type symbols to instances of media type classes. These classes determine how the response is processed.

When using the media type option:


Next: , Up: Usage   [Contents]

2.1 Quick Start

The plz-media-types variable contains a list of pre-configured media type mappings. The following code makes a synchronous GET request to a URL and returns a plz-response structure. Since the response is returned with the content type header set to application/json, the plz-media-type:application/json class will be used to decode the HTTP body.

(plz-media-type-request
  'get "https://httpbin.org/json"
  :as `(media-types ,plz-media-types))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:41:18 GMT")
                 (content-type . "application/json")
                 (content-length . "429")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                ((slideshow
                  (author . "Yours Truly")
                  (date . "date of publication")
                  (slides .
                          [((title . "Wake up to WonderWidgets!")
                            (type . "all"))
                           ((items .
                                   ["Why <em>WonderWidgets</em> are great" "Who <em>buys</em> WonderWidgets"])
                            (title . "Overview")
                            (type . "all"))])
                  (title . "Sample Slide Show"))))

Previous: , Up: Usage   [Contents]

2.2 Media Types

The following table shows the media type classes provided by the plz-media-type library. The Content Type and Default columns indicate which media type is part of the default plz-media-types and which content type header values trigger their use. The Stream column specifies whether the class consumes data in a streaming or non-streaming way.

EIEIO ClassContent TypeDefaultStream
plz-media-type:application/json-array-NoYes
plz-media-type:application/jsonapplication/jsonYesNo
plz-media-type:application/octet-streamapplication/octet-streamYesNo
plz-media-type:application/x-ndjson-NoYes
plz-media-type:application/xmlapplication/xmlYesNo
plz-media-type:text/htmltext/htmlYesNo

2.2.1 JSON

The plz-media-type:application/json media type class handles JSON responses. It parses the response body using json-parse-buffer after receiving the entire response.

Customization:

Instances of this class can be customized by setting the following slots:

  • :array-type: Lisp type used for arrays (array or list, default array)
  • :false-object: Object representing JSON false (default :json-false)
  • :null-object: Object representing JSON null (default nil)
  • :object-type: Lisp type used for objects (hash-table, alist, or plist, default alist)

Example:

(plz-media-type-request
  'get "https://httpbin.org/json"
  :as `(media-types
        ((application/json
          . ,(plz-media-type:application/json :array-type 'list)))))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:05:19 GMT")
                 (content-type . "application/json")
                 (content-length . "429")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                ((slideshow
                  (author . "Yours Truly")
                  (date . "date of publication")
                  (slides
                   ((title . "Wake up to WonderWidgets!")
                    (type . "all"))
                   ((items "Why <em>WonderWidgets</em> are great" "Who <em>buys</em> WonderWidgets")
                    (title . "Overview")
                    (type . "all")))
                  (title . "Sample Slide Show"))))

Next: , Previous: , Up: Media Types   [Contents]

2.2.2 Newline Delimited JSON Stream

The plz-media-type:application/x-ndjson media type class handles newline-delimited JSON responses in a streaming fashion. It parses each line using json-parse-buffer and calls the :handler function for each decoded object. The body slot of the plz-response structure will always be nil.

Customization:

  • :handler: Function called for each JSON object.

Example:

(plz-media-type-request
  'get "https://httpbin.org/stream/5"
  :as `(media-types
        ((application/json
          . ,(plz-media-type:application/x-ndjson
              :handler (lambda (object)
                         (message "%s" object)))))))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:06:51 GMT")
                 (content-type . "application/json")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                nil)

2.2.3 JSON Array Stream

The plz-media-type:application/json-array media type class handles JSON responses where the top-level object is an array. It parses each object in the array using json-parse-buffer and calls the :handler function for each decoded object.

Customization:

  • :handler: Function called for each JSON object.

Example:

(plz-media-type-request
  'get "https://localhost/json-array"
  :as `(media-types
        ((application/json
          . ,(plz-media-type:application/json-array
              :handler (lambda (object)
                         (message "%s" object)))))))

Next: , Previous: , Up: Media Types   [Contents]

2.2.4 XML

The plz-media-type:application/xml media type class handles XML responses. It parses the response body using libxml-parse-html-region after receiving the entire response.

Customization: None

Example:

(plz-media-type-request
  'get "https://httpbin.org/xml"
  :as `(media-types ((application/xml . ,(plz-media-type:application/xml)))))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:17:57 GMT")
                 (content-type . "application/xml")
                 (content-length . "522")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                (top nil
                     (comment nil "  A SAMPLE set of slides  ")
                     (html nil
                           (body nil
                                 (slideshow
                                  ((title . "Sample Slide Show")
                                   (date . "Date of publication")
                                   (author . "Yours Truly"))
                                  (comment nil " TITLE SLIDE ")
                                  (slide
                                   ((type . "all"))
                                   (title nil "Wake up to WonderWidgets!"))
                                  (comment nil " OVERVIEW ")
                                  (slide
                                   ((type . "all"))
                                   (title nil "Overview")
                                   (item nil "Why "
                                         (em nil "WonderWidgets")
                                         " are great")
                                   (item nil)
                                   (item nil "Who "
                                         (em nil "buys")
                                         " WonderWidgets")))))))

Next: , Previous: , Up: Media Types   [Contents]

2.2.5 HTML

The plz-media-type:text/html media type class handles HTML responses. It parses the response body using libxml-parse-html-region after receiving the entire response.

Customization: None

Example:

(plz-media-type-request
  'get "https://httpbin.org/html"
  :as `(media-types ((text/html . ,(plz-media-type:application/xml)))))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:18:40 GMT")
                 (content-type . "text/html; charset=utf-8")
                 (content-length . "3741")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                (html nil
                      (head nil)
                      (body nil "\n      "
                            (h1 nil "Herman Melville - Moby-Dick")
                            "\n\n      "
                            (div nil "\n        "
                                 (p nil "\n          Availing himself of the mild, summer-cool weather that now reigned in these latitudes, and in preparation for the peculiarly active pursuits shortly to be anticipated, Perth, the begrimed, blistered old blacksmith, had not removed his portable forge to the hold again, after concluding his contributory work for Ahab's leg, but still retained it on deck, fast lashed to ringbolts by the foremast; being now almost incessantly invoked by the headsmen, and harpooneers, and bowsmen to do some little job for them; altering, or repairing, or new shaping their various weapons and boat furniture. Often he would be surrounded by an eager circle, all waiting to be served; holding boat-spades, pike-heads, harpoons, and lances, and jealously watching his every sooty movement, as he toiled. Nevertheless, this old man's was a patient hammer wielded by a patient arm. No murmur, no impatience, no petulance did come from him. Silent, slow, and solemn; bowing over still further his chronically broken back, he toiled away, as if toil were life itself, and the heavy beating of his hammer the heavy beating of his heart. And so it was.—Most miserable! A peculiar walk in this old man, a certain slight but painful appearing yawing in his gait, had at an early period of the voyage excited the curiosity of the mariners. And to the importunity of their persisted questionings he had finally given in; and so it came to pass that every one now knew the shameful story of his wretched fate. Belated, and not innocently, one bitter winter's midnight, on the road running between two country towns, the blacksmith half-stupidly felt the deadly numbness stealing over him, and sought refuge in a leaning, dilapidated barn. The issue was, the loss of the extremities of both feet. Out of this revelation, part by part, at last came out the four acts of the gladness, and the one long, and as yet uncatastrophied fifth act of the grief of his life's drama. He was an old man, who, at the age of nearly sixty, had postponedly encountered that thing in sorrow's technicals called ruin. He had been an artisan of famed excellence, and with plenty to do; owned a house and garden; embraced a youthful, daughter-like, loving wife, and three blithe, ruddy children; every Sunday went to a cheerful-looking church, planted in a grove. But one night, under cover of darkness, and further concealed in a most cunning disguisement, a desperate burglar slid into his happy home, and robbed them all of everything. And darker yet to tell, the blacksmith himself did ignorantly conduct this burglar into his family's heart. It was the Bottle Conjuror! Upon the opening of that fatal cork, forth flew the fiend, and shrivelled up his home. Now, for prudent, most wise, and economic reasons, the blacksmith's shop was in the basement of his dwelling, but with a separate entrance to it; so that always had the young and loving healthy wife listened with no unhappy nervousness, but with vigorous pleasure, to the stout ringing of her young-armed old husband's hammer; whose reverberations, muffled by passing through the floors and walls, came up to her, not unsweetly, in her nursery; and so, to stout Labor's iron lullaby, the blacksmith's infants were rocked to slumber. Oh, woe on woe! Oh, Death, why canst thou not sometimes be timely? Hadst thou taken this old blacksmith to thyself ere his full ruin came upon him, then had the young widow had a delicious grief, and her orphans a truly venerable, legendary sire to dream of in their after years; and all of them a care-killing competency.\n        ")
                                 "\n      ")
                            "\n  ")))

Previous: , Up: Media Types   [Contents]

2.2.6 Octet Stream

The plz-media-type:application/octet-stream media type class handles any other response and is used as the default media type handler. It does not parse the response body in any way.

(plz-media-type-request
  'get "https://httpbin.org/json"
  :as `(media-types ((t . ,(plz-media-type:application/octet-stream)))))
#s(plz-response 2 200
                ((date . "Sun, 24 Mar 2024 10:28:40 GMT")
                 (content-type . "application/json")
                 (content-length . "429")
                 (server . "gunicorn/19.9.0")
                 (access-control-allow-origin . "*")
                 (access-control-allow-credentials . "true"))
                "{\n  \"slideshow\": {\n    \"author\": \"Yours Truly\", \n    \"date\": \"date of publication\", \n    \"slides\": [\n      {\n        \"title\": \"Wake up to WonderWidgets!\", \n        \"type\": \"all\"\n      }, \n      {\n        \"items\": [\n          \"Why <em>WonderWidgets</em> are great\", \n          \"Who <em>buys</em> WonderWidgets\"\n        ], \n        \"title\": \"Overview\", \n        \"type\": \"all\"\n      }\n    ], \n    \"title\": \"Sample Slide Show\"\n  }\n}\n")

Next: , Previous: , Up: plz-media-type   [Contents]

3 Troubleshooting


3.1 Debugging HTTP responses

It might be useful to see the full HTTP response while developing a new media type format or plz-media-type is failing to parse a response of an HTTP request. When the plz-media-type-debug-p custom variable is set to a non-nil value, each chunk of the HTTP response is written to the buffer configured by the plz-media-type-debug-response-buffer custom variable as it arrives. The buffer is cleared on each new request, unless plz-media-type-debug-erase-buffer-p is set to a non-nil value.


4 Credits


6 License

GPLv3