| ... | ... | @@ -10447,13 +10447,40 @@ fn readU32Be() u32 {} |
| 10447 | 10447 | {#header_close#} |
| 10448 | 10448 | {#header_open|Source Encoding#} |
| 10449 | 10449 | <p>Zig source code is encoded in UTF-8. An invalid UTF-8 byte sequence results in a compile error.</p> |
| 10450 | | <p>Throughout all zig source code (including in comments), some codepoints are never allowed:</p> |
| 10450 | <p>Throughout all zig source code (including in comments), some code points are never allowed:</p> |
| 10451 | 10451 | <ul> |
| 10452 | | <li>Ascii control characters, except for U+000a (LF): U+0000 - U+0009, U+000b - U+0001f, U+007f. (Note that Windows line endings (CRLF) are not allowed, and hard tabs are not allowed.)</li> |
| 10452 | <li>Ascii control characters, except for U+000a (LF), U+000d (CR), and U+0009 (HT): U+0000 - U+0008, U+000b - U+000c, U+000e - U+0001f, U+007f.</li> |
| 10453 | 10453 | <li>Non-Ascii Unicode line endings: U+0085 (NEL), U+2028 (LS), U+2029 (PS).</li> |
| 10454 | 10454 | </ul> |
| 10455 | | <p>The codepoint U+000a (LF) (which is encoded as the single-byte value 0x0a) is the line terminator character. This character always terminates a line of zig source code (except possibly the last line of the file).</p> |
| 10456 | | <p>For some discussion on the rationale behind these design decisions, see <a href="https://github.com/ziglang/zig/issues/663">issue #663</a></p> |
| 10455 | <p> |
| 10456 | LF (byte value 0x0a, code point U+000a, {#syntax#}'\n'{#endsyntax#}) is the line terminator in Zig source code. |
| 10457 | This byte value terminates every line of zig source code except the last line of the file. |
| 10458 | It is recommended that non-empty source files end with an empty line, which means the last byte would be 0x0a (LF). |
| 10459 | </p> |
| 10460 | <p> |
| 10461 | Each LF may be immediately preceded by a single CR (byte value 0x0d, code point U+000d, {#syntax#}'\r'{#endsyntax#}) |
| 10462 | to form a Windows style line ending, but this is discouraged. |
| 10463 | A CR in any other context is not allowed. |
| 10464 | </p> |
| 10465 | <p> |
| 10466 | HT hard tabs (byte value 0x09, code point U+0009, {#syntax#}'\t'{#endsyntax#}) are interchangeable with |
| 10467 | SP spaces (byte value 0x20, code point U+0020, {#syntax#}' '{#endsyntax#}) as a token separator, |
| 10468 | but use of hard tabs is discouraged. See {#link|Grammar#}. |
| 10469 | </p> |
| 10470 | <p> |
| 10471 | Note that running <code>zig fmt</code> on a source file will implement all recommendations mentioned here. |
| 10472 | Note also that the stage1 compiler does <a href="https://github.com/ziglang/zig/wiki/FAQ#why-does-zig-force-me-to-use-spaces-instead-of-tabs">not yet support CR or HT</a> control characters. |
| 10473 | </p> |
| 10474 | <p> |
| 10475 | Note that a tool reading Zig source code can make assumptions if the source code is assumed to be correct Zig code. |
| 10476 | For example, when identifying the ends of lines, a tool can use a naive search such as <code>/\n/</code>, |
| 10477 | or an <a href="https://msdn.microsoft.com/en-us/library/dd409797.aspx">advanced</a> |
| 10478 | search such as <code>/\r\n?|[\n\u0085\u2028\u2029]/</code>, and in either case line endings will be correctly identified. |
| 10479 | For another example, when identifying the whitespace before the first token on a line, |
| 10480 | a tool can either use a naive search such as <code>/[ \t]/</code>, |
| 10481 | or an <a href="https://tc39.es/ecma262/#sec-characterclassescape">advanced</a> search such as <code>/\s/</code>, |
| 10482 | and in either case whitespace will be correctly identified. |
| 10483 | </p> |
| 10457 | 10484 | {#header_close#} |
| 10458 | 10485 | |
| 10459 | 10486 | {#header_open|Keyword Reference#} |
| ... | ... | @@ -11373,6 +11400,7 @@ ExprList <- (Expr COMMA)* Expr? |
| 11373 | 11400 | |
| 11374 | 11401 | # *** Tokens *** |
| 11375 | 11402 | eof <- !. |
| 11403 | eol <- ('\r'? '\n') | eof |
| 11376 | 11404 | hex <- [0-9a-fA-F] |
| 11377 | 11405 | hex_ <- ('_'/hex) |
| 11378 | 11406 | dec <- [0-9] |
| ... | ... | @@ -11382,39 +11410,39 @@ dec_int <- dec (dec_* dec)? |
| 11382 | 11410 | hex_int <- hex (hex_* dec)? |
| 11383 | 11411 | |
| 11384 | 11412 | char_escape |
| 11385 | | <- "\\x" hex hex |
| 11386 | | / "\\u{" hex+ "}" |
| 11387 | | / "\\" [nr\\t'"] |
| 11413 | <- '\\x' hex hex |
| 11414 | / '\\u{' hex+ '}' |
| 11415 | / '\\' [nr\\t'"] |
| 11388 | 11416 | char_char |
| 11389 | 11417 | <- char_escape |
| 11390 | | / [^\\'\n] |
| 11418 | / [^\\'\r\n] |
| 11391 | 11419 | string_char |
| 11392 | 11420 | <- char_escape |
| 11393 | | / [^\\"\n] |
| 11421 | / [^\\"\r\n] |
| 11394 | 11422 | |
| 11395 | | line_comment <- '//'[^\n]* |
| 11396 | | line_string <- ("\\\\" [^\n]* [ \n]*)+ |
| 11397 | | skip <- ([ \n] / line_comment)* |
| 11423 | line_comment <- '//'[^\r\n]* eol |
| 11424 | line_string <- ('\\\\' [^\r\n]* eol skip)+ |
| 11425 | skip <- ([ \t] / eol / line_comment)* |
| 11398 | 11426 | |
| 11399 | 11427 | CHAR_LITERAL <- "'" char_char "'" skip |
| 11400 | 11428 | FLOAT |
| 11401 | | <- "0x" hex_* hex "." hex_int ([pP] [-+]? hex_int)? skip |
| 11402 | | / dec_int "." dec_int ([eE] [-+]? dec_int)? skip |
| 11403 | | / "0x" hex_* hex "."? [pP] [-+]? hex_int skip |
| 11404 | | / dec_int "."? [eE] [-+]? dec_int skip |
| 11429 | <- '0x' hex_* hex '.' hex_int ([pP] [-+]? hex_int)? skip |
| 11430 | / dec_int '.' dec_int ([eE] [-+]? dec_int)? skip |
| 11431 | / '0x' hex_* hex '.'? [pP] [-+]? hex_int skip |
| 11432 | / dec_int '.'? [eE] [-+]? dec_int skip |
| 11405 | 11433 | INTEGER |
| 11406 | | <- "0b" [_01]* [01] skip |
| 11407 | | / "0o" [_0-7]* [0-7] skip |
| 11408 | | / "0x" hex_* hex skip |
| 11434 | <- '0b' [_01]* [01] skip |
| 11435 | / '0o' [_0-7]* [0-7] skip |
| 11436 | / '0x' hex_* hex skip |
| 11409 | 11437 | / dec_int skip |
| 11410 | | STRINGLITERALSINGLE <- "\"" string_char* "\"" skip |
| 11438 | STRINGLITERALSINGLE <- '"' string_char* '"' skip |
| 11411 | 11439 | STRINGLITERAL |
| 11412 | 11440 | <- STRINGLITERALSINGLE |
| 11413 | | / line_string skip |
| 11441 | / line_string skip |
| 11414 | 11442 | IDENTIFIER |
| 11415 | 11443 | <- !keyword [A-Za-z_] [A-Za-z0-9_]* skip |
| 11416 | | / "@\"" string_char* "\"" skip |
| 11417 | | BUILTINIDENTIFIER <- "@"[A-Za-z_][A-Za-z0-9_]* skip |
| 11444 | / '@"' string_char* '"' skip |
| 11445 | BUILTINIDENTIFIER <- '@'[A-Za-z_][A-Za-z0-9_]* skip |
| 11418 | 11446 | |
| 11419 | 11447 | |
| 11420 | 11448 | AMPERSAND <- '&' ![=] skip |
Josh Wolfe <thejoshwolfe@gmail.com> 2021-03-03 14:30:46-05:00
GitHub <noreply@github.com> 2021-03-03 14:30:46-05:00