Normative Rules: Source-File and File-Stream Encoding

← Prev | ↑ Chapter | Next → | Index | Symbols

Normative Rules: Source-File and File-Stream Encoding

Statement

AutoLISP source files and host file streams have two independent encoding axes:

  1. Source-file encoding — the encoding the runtime uses when load (and any reader entry point that takes a filesystem path) reads bytes from disk. Historically AutoLISP ran on Windows with ANSI / MBCS source files. On AutoCAD 2021 and later the default is decided by the host system variable LISPSYS (see LISPSYS governs the AutoCAD source default below): 0 keeps the legacy ANSI/MBCS reader (the Windows ANSI code page, windows-1252 in the western locales), 1 and 2 select Unicode (UTF-8) and accept the optional encoding argument on open. A new installation has LISPSYS = 1; users of windows-1252 code bases set it to 0. AutoCAD before 2021 has no LISPSYS and is always ANSI/MBCS. LISPSYS is saved in the registry and requires an AutoCAD restart to take effect.
  2. File-stream encoding — the encoding the runtime uses when open reads or writes the bytes of a host file the user is doing I/O on. This is the third (optional) argument to open. When the argument is omitted, the per-host default applies — ANSI/MBCS for the legacy engine, UTF-8 for the modern engine.

There is no documented per-string encoding API: AutoLISP strings are sequences of code units in the host engine's native domain (8-bit code page for the legacy engine, Unicode-capable host string for the modern engine). String content is exchanged with files through the encoding configured for that file's stream.

Encoding Names

The third argument to open accepts a small documented vocabulary. Implementations should additionally accept the listed aliases since deployed source uses them interchangeably:

Designator Maps to (CL external-format)
"utf8" :utf-8
"utf8-bom" :utf-8 with byte-order mark
"utf16" :utf-16
"ANSI" :iso-8859-1 (the conservative 1-1
  mapping that never fails on bytes)
"ASCII" :ascii
"latin1" / "iso-8859-1" :iso-8859-1
"cp1252" / "windows-1252" :cp1252

Hosts derived from the BricsCAD codebase additionally accept the C-style ccs=NAME syntax embedded in the second (mode) argument, e.g. (open path "r,ccs=UTF-8"). The clautolisp bricscad-v26 dialect honours this through the existing open-ccs-mode-p knob; the strict and autocad-2026 dialects do not.

Per-Dialect Defaults

clautolisp resolves the absent third argument of open — and the absent :external-format keyword on load — using the active dialect descriptor:

Dialect Default source encoding Default file encoding
:strict :iso-8859-1 :iso-8859-1
AutoCAD 2021+ (autocad, autocad-2022, autocad-2026 …) LISPSYS 0: :cp1252; 1 / 2: :utf-8 same as source
AutoCAD before 2021 :cp1252 (ANSI) :cp1252
:bricscad-v26 :utf-8 :utf-8

(Implementation deviation for the AutoCAD 2021+ row: see Implementation deviation: OPEN's default under the AutoCAD dialects below.)

The strict choice is deliberately permissive at the byte level: a 1-1 byte coding never raises a decoding error on any input, and existing AutoLISP corpora produced under the legacy ANSI/MBCS engine remain loadable. Source code that needs Unicode characters in literals can either (a) use the explicit --dialect autocad-2026 / --bricscad-v26 flag, or (b) pass :external-format :utf-8 on the call site.

Vendor Evidence

  • Autodesk reference for open (AutoCAD 2024+) documents "utf8" and "utf8-bom" as the only accepted encoding strings; "When a value isn't provided for the argument, the file is assumed to contain multibyte character set (MBCS) which is the legacy behavior."
  • Autodesk LISPSYS sysvar reference: 0 = ASCII / no encoding arg, 1 = Unicode, 2 = Unicode-with-extra-COM. Restart required.
  • (Superseded as the rule for the default: an Autodesk Community post quoting AutoCAD 2025, "In the previous versions of AutoCAD, the encoding of the file was 'ANSI', now it is 'UTF-8'", describes files the Visual LISP IDE writes; the default for READING is governed by LISPSYS since 2021 – see below.)
  • BricsCAD V26 honours the C-runtime ccs=NAME embedded mode-string as documented in earlier Phase-6 probe evidence (open-ccs-mode-p).

LISPSYS governs the AutoCAD source default (2021 and later)

Reference. Autodesk, LISPSYS system variable (AutoCAD 2021): type integer, saved in the REGISTRY, initial value 1. "0: … AutoLISP functions don't fully support Unicode … source files … use the ASCII (MBCS) character set … behavior of AutoCAD 2020 and earlier releases"; "1: … fully support Unicode"; "2: … Unicode, … when compiled … ASCII (MBCS)". A restart is required for a change to take effect. "Starting with AutoCAD 2021-based products, AutoLISP source files can be saved in either ASCII or Unicode file formats … AutoCAD 2020-based and earlier products, only support the ASCII file format."

Measured. AutoCAD 2022 / accoreconsole on the CI's Windows machine, whose LISPSYS is 0 (probes/sources/probe-versions.lsp, job 16921684058): source and open "w" in cp1252 (the encoding experiments above); with 1 / 2 the reader and writer are UTF-8.

Rule (normative for the AutoCAD dialects). On AutoCAD 2021 and later the default source-file encoding – and the default of open without an encoding argument – is the one LISPSYS selects AT STARTUP: 0, the ANSI code page (windows-1252); 1 or 2, UTF-8. A (setvar "LISPSYS" n) is saved and governs the NEXT session, not the current one. Before 2021 the default is the ANSI code page.

Implementation deviation: OPEN's default under the AutoCAD dialects

The Rule above and the Per-Dialect Defaults table are kept as written (pjb, 2026-10-07: the normative text stays as in the reference documents). What AutoCAD 2022 does, measured (experiment E1, job 16980926802, probe lispsys-bom-probe.lsp), and what clautolisp implements since 2.2.222, differ from them for open at LISPSYS 1 / 2:

open call, LISPSYS 1 / 2 Rule / table AutoCAD 2022, measured = clautolisp 2.2.222
(open f "w"), no encoding UTF-8 ("same as source") windows-1252 (MBCS): 3 bytes for "A" e-acute "B"
(open f "w" "utf8-bom") UTF-8 with byte-order mark BOM + UTF-8 (7 bytes) – agrees
(open f "r"), no encoding UTF-8 reads a windows-1252, a UTF-8 and a BOM+UTF-8 file alike (BOM skipped, cp1252 fallback)
(open f "r" "utf8") UTF-8 strict: returns U+FEFF of a BOM, stops at the first invalid byte

So the write default does NOT follow LISPSYS – it stays the MBCS code page Autodesk's open page describes for an absent encoding – while the read default does, leniently. At LISPSYS 0 AutoCAD rejects any third argument ("too many arguments"); clautolisp accepts it with the dialect warning (pjb, 2026-10-06). Details, and the points the measurement left open, in the *** clautolisp section of Function Entry: OPEN and in issues/closed/autocad-open-encoding-lispsys-divergence.issue.

clautolisp: LISPSYS, persisted and read at launch

clautolisp keeps LISPSYS where the vendor does – in the host's persistent registry store (the Windows registry, the macOS defaults database, the XDG configuration file elsewhere; see vl-registry-read), under HKEY_CURRENT_USER\Software\clautolisp\Variables\<product>, value LISPSYS – and per product (an AutoCAD dialect's value does not leak into a BricsCAD one). (setvar "LISPSYS" n) writes it; every launch reads it back (default 1 for AutoCAD, BricsCAD's measured 0 for BricsCAD) and, under the AutoCAD 2021+ dialects, derives the default source and file encoding from it, ahead of the POSIX locale (AutoCAD does not consult the locale) – for load and open's reads: open's default WRITE stays windows-1252 at every level, as measured on AutoCAD 2022 (clautolisp 2.2.222; see the *** clautolisp section of Function Entry: OPEN). An explicit -Esource / -e still wins: the user asked for that encoding. The :strict, :clautolisp, :lax and BricsCAD dialects keep their encoding defaults; BricsCAD's own LISPSYS (V23.1: "allows to use UTF-8 (without BOM header) encoded Lisp files") is recorded, not yet acted on.

Per-open, not per-read

The encoding is fixed when the stream is opened, not chosen per read-char / read-line / read (or per princ / write-line). AutoCAD and BricsCAD both bind the decoder at open time — through the third argument or the ccs= mode suffix — and subsequent reads on the same handle cannot change it. clautolisp matches this: open resolves the external-format once (from its explicit argument, then *AUTOLISP-FILE-ENCODING*, then the dialect default) and holds it for the life of the stream. A program that must read parts of a file under different encodings must close and re-open it.

Extension (clautolisp): *AUTOLISP-FILE-ENCODING*

clautolisp adds a settable global, *AUTOLISP-FILE-ENCODING* — a string, where the empty string "" means "no preference; fall through to the dialect default". It is consulted when neither an explicit open third argument nor a load :external-format is supplied, and applies to BOTH directions: the read side of load / open "r" / read-line, and the write side of open "w"/"a" + princ / write-line. A pure-AutoLISP program can therefore author a file and read it back in a chosen encoding with no CLI flag. The front-end sets it from -Esource ENC (alfe) / -e ENC; it may be reassigned at runtime between open calls (not mid-stream — see the per-open rule above). This variable does not exist on AutoCAD or BricsCAD; portable code selects the encoding through the open third argument / ccs= instead. Status: clautolisp-only extension.

Measured vendor divergences (2026-07, clautolisp probe evidence)

Observed behaviours on specific (product × version × platform) targets, recorded so the divergence taxonomy (chapter 25) can classify the feature. They are not normative for the language; portable code must not rely on them. The front-end's full (situation × os × tool × version) defaults table with provenance lives in the alfe integration notes.

  • AutoCAD / accoreconsole 2022 (Windows), LISPSYS 0 — open ignores the ccs= suffix; the file-write default is ANSI1252 / MBCS, and a non-Latin-1 char is written as its low byte (U+20AC → 0xAC). LISPSYS 1/2 switch the reader/writer to UTF-8 (restart-level; LISPSYS selects cp1252 vs UTF-8). The accoreconsole console pipe is UTF-16LE, fixed by the product — no /l language flag or code page changes it.
  • BricsCAD V26 (macOS) — the open "w" default follows the POSIX locale, not SYSCODEPAGE (which reads MAC_ROMAN): a UTF-8 locale writes UTF-8, LC_ALL=C writes Latin-1. On READ, a UTF-8 file is decoded only with a BOM — a BOM-less UTF-8 file reads back as raw bytes even under ,ccs=UTF-8. ,ccs=UTF-16LE is accepted but silently writes UTF-8 — a platform divergence: on BricsCAD/Windows it writes real UTF-16LE.
  • BricsCAD V25 (Windows) — open "w" default is ANSI1252; both ,ccs=UTF-8 and ,ccs=UTF-16LE are honoured on write.
  • GUI console encodings — BricsCAD/macOS GUI console is full Unicode both ways (é = 233, € = 8364); the Windows GUI CADs (BricsCAD, AutoCAD) are cp1252 (é = 233, € = 128), AutoCAD truncating € to 0xAC on output.
  • Native (load) decode — the encoding a CAD's own (load path) uses to decode a source file follows the same (product × version × platform × LISPSYS) axis; the clautolisp front-end forwards an explicit -Esource into the CAD's per-open spelling where the engine supports it.

Vendor-documented load / open encoding rules (normative reference)

These are the concrete, version-qualified rules the vendors publish; the whole macOS cp1252 problem (a source file's (chr N) diverging from the same character read from disk) follows from them.

  • load — no encoding argument. Signature (load lspfile [failure-expr]); extension search name.des then name.lsp when none is given. Since BricsCAD V20 (and AutoCAD 2021 under LISPSYS 1/2) load also reads Unicode Lisp files stored as UTF-8 (Little-Endian) or UTF-16 (Little-Endian) — but the Byte-Order-Mark (BOM) is mandatory: a UTF-8 file without a BOM may fail. With no BOM the file is decoded with the host SYSCODEPAGE, which is read-only (e.g. MAC_ROMAN on macOS, ansi_1252 on French Windows). Consequence: a raw single-byte-codepage (e.g. cp1252) source cannot be portably native-loaded — on macOS SYSCODEPAGE = MAC_ROMAN mis-reads byte 0xE9 as È (U+00C8) instead of é (U+00E9). Measured on AutoCAD 2022 (encoding-situations experiment E1, 2026-10-05, job 16931781178; LISPSYS set in the profile, AutoCAD restarted per level; a file (setq v "AéZ") with é written three ways, (vl-string->list v)):

    file LISPSYS 0 LISPSYS 1 / 2
    windows-1252 (E9) (65 233 90) (65 233 90)
    UTF-8, no BOM (65 195 169 90) (65 233 90)
    UTF-8 with BOM (65 195 169 90) (65 233 90)

    So the BOM is NOT mandatory under LISPSYS 1 / 2 (BOM-less UTF-8 reads fine), a file that is not valid UTF-8 FALLS BACK to windows-1252 instead of failing, and under LISPSYS 0 the BOM is skipped and ignored. clautolisp: under the AutoCAD dialects, a load without an explicit encoding follows this table (the encoding LISPSYS selected, then the BOM skip at windows-1252 or the windows-1252 fallback at UTF-8); an explicit encoding – load's third argument – is honoured as given.

  • open — optional encoding argument (AutoCAD 2021, BricsCAD V21 and higher; pre-2021 open has none). Signature (open fileName openMode [encoding]); modes "r" "w" "a" plus binary "rb" "wb" "ab". The documented encoding values are "utf8", "utf8-bom", "utf16-bom" — "UTF-8 without BOM can not be distinguished from normal ANSI". (BricsCAD additionally accepts a "MODE,ccs=ENC" mode spelling for open; ccs= is an open-only knob and has no load equivalent.)
  • Canonical-representation contract (normative) — whatever the file encoding, a decoded non-ASCII character MUST satisfy (= (chr N) <same char decoded from a file>): one canonical internal representation per character. clautolisp already guarantees this; BricsCAD's native chr is correct ((chr 233) = é) but a file char decoded with the wrong codec is not, so the two diverge — a front-end/host contract, not a chr bug. See issues/closed/cad-chr-vs-loaded-encoding.issue.
  • alfe consequence — because load takes no encoding and SYSCODEPAGE is not settable, a -Esource ENC whose ENC is neither the host SYSCODEPAGE nor a BOM'd UTF-8/UTF-16 must be converted before the CAD loads it: alfe decodes the source with ENC and stages a UTF-8-with-BOM copy, which every platform's load detects and decodes correctly.

Cross-references

  • Special Form Entry: LOAD (chapter 11) — uses the dialect's source-encoding default.
  • Function Entry: OPEN (chapter 11) — uses the dialect's file-encoding default when its third argument is omitted.
  • alfe integration: the -E{situation} encoding CLI family and the (situation × os × tool × version) defaults table with provenance (source / file / console / cadstdio / log / terminal).