View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0000733luatexluatex bugpublic2011-11-16 00:122015-03-26 18:21
Reporteroneiros Assigned ToHans Hagen  
PrioritynormalSeveritymajorReproducibilityalways
Status closedResolutionfixed 
Product Version0.70.1 
Summary0000733: \scantokens behaves differently than in pdfetex
DescriptionThe implementation of \scantokens seems to be broken; the attached file works perfectly with pdfTeX 1.40.12, but fails with luatex 0.70.1-2011061410 (rev 4277)
Steps To Reproduce> etex --ini --etex t.tex
This is pdfTeX, Version 3.1415926-2.3-1.40.12 (TeX Live 2011) (INITEX)
 restricted \write18 enabled.
entering extended mode
(./t.tex
macro:->a b
 )
No pages of output.
Transcript written on t.log.
> luatex --ini --etex t.tex
This is LuaTeX, Version beta-0.70.1-2011061410 (rev 4277) (INITEX)
 restricted \write18 enabled.
(./t.tex
Runaway definition?
->a
! File ended while scanning definition of \x.
<inserted text>
                }
l.9 \edef\x{\scantokens{a
b\noexpand}
                                     }
? x
No pages of output.
Transcript written on t.log.
Additional InformationThis is discussed on the lualatex-dev list:
http://tug.org/pipermail/lualatex-dev/2011-November/001376.html
TagsNo tags attached.

Activities

oneiros

2011-11-16 00:12

developer  

t.tex (228 bytes)    
\catcode`\{=1
\catcode`\}=2
\ifx\directlua\undefined
\else
 \directlua{tex.enableprimitives('',{'scantokens'})}
\fi

\newlinechar=`q
\edef\x{\scantokens{aqb\noexpand}}
\immediate\write16{\meaning\x}

\csname @@end\endcsname\end
t.tex (228 bytes)   

Hans Hagen

2015-03-25 17:17

manager   ~0001349

These things depends on the macro package and how it defines \obeylines. It also depends on how \everyeof is defined etc. And when you end up with $<nothing>$ the regular message is displayed.

\newtoks\mytoks

\begingroup
\catcode 13 13
\xdef
{!}%
\global\mytoks{
}%
\xdef\xx{\the\mytoks}[\xx]%
\everymath{}%
\everyeof{?}%
\tracingall%
($<\scantokens\expandafter{\expandafter$\the\mytoks$}>$)%
($<\scantextokens\expandafter{\expandafter$\the\mytoks$}>$)%
\endgroup%

Issue History

Date Modified Username Field Change
2011-11-16 00:12 oneiros New Issue
2011-11-16 00:12 oneiros File Added: t.tex
2015-03-25 17:17 Hans Hagen Note Added: 0001349
2015-03-26 18:21 Hans Hagen Status new => closed
2015-03-26 18:21 Hans Hagen Assigned To => Hans Hagen
2015-03-26 18:21 Hans Hagen Resolution open => fixed