View Issue Details

IDProjectCategoryView StatusLast Update
0000095luatexfeature requestpublic2015-10-21 23:42
ReporterJonathan Sauer Assigned ToHans Hagen  
Status closedResolutionfixed 
Product Version0.29.0 
Target Version0.80.0Fixed in Version0.82.0 
Summary0000095: Possibility to specify catcode table when setting a toks register from Lua
DescriptionCurrently, when setting a toks register from Lua using tex.toks[...] = ... or tex.settoks(...), the catcodes of the characters are set to 10 (spaces) and 12 (everything else), respectively. This makes it difficult to pass data from Lua to TeX using toks registers, since i.e. control sequences are \string-ified.

It would be great if the mechanism for setting toks registers was extended, so that an optional catcode table could be specified, i.e:

 tex.settoks(<catcode table>, <string>)

This catcode table would then be used for tokenizing the Lua string.
TagsNo tags attached.



2008-10-04 11:27

administrator   ~0000072

Actually, the toks registers will move away from pure strings and start using tokenlist tables instead. At the same time, there will be a converter from strings to tokenlists that will indeed allow setting an explicit catcode table.

So you will get something like

   tex.toks[0] = tex.scan_toks (<catcode table>, "....")

Sounds reasonable?

Jonathan Sauer

2008-10-07 10:25

reporter   ~0000078

It does indeed, although I'm a bit worried about performance, since if I understand correctly the tokenlist table will be purely temporary and converted into another data structure internal to TeX.

Hans Hagen

2015-10-21 23:42

manager   ~0001463

implemented: tex.scantoks("global",register,cct,"string")

Issue History

Date Modified Username Field Change
2008-09-30 12:04 Jonathan Sauer New Issue
2008-10-04 11:27 Taco Note Added: 0000072
2008-10-07 10:25 Jonathan Sauer Note Added: 0000078
2010-01-12 09:53 Taco Assigned To => Taco
2010-01-12 09:53 Taco Status new => assigned
2010-01-12 09:53 Taco Target Version => 0.60.0
2010-03-13 09:36 Taco Target Version 0.60.0 => 0.70.0
2011-04-12 11:09 Taco Target Version 0.70.0 => 0.80.0
2015-10-21 23:41 Hans Hagen Assigned To Taco => Hans Hagen
2015-10-21 23:42 Hans Hagen Note Added: 0001463
2015-10-21 23:42 Hans Hagen Status assigned => closed
2015-10-21 23:42 Hans Hagen Resolution open => fixed
2015-10-21 23:42 Hans Hagen Fixed in Version => 0.82.0