#### View Issue Details

ID Project Category View Status Date Submitted Last Update 0000937 luatex luatex bug public 2015-05-11 18:32 2015-11-02 17:40 pdfkungfoo Hans Hagen normal minor always closed won't fix Tex Live 2014 OS X Mavericks 10.9.5 0.80.1 maybe never (> v 1) 0000937: lualatex trips over some JPEGS and aborts PDF generating process (or leads to unexpected results) The lualatex command trips over some JPEG files and aborts the PDF generating process. Here is an MWE, mini.tex: ############################## \documentclass[a4paper]{article} \usepackage{graphicx} \begin{document} \thispagestyle{empty} \begin{figure}[htbp] \centering \includegraphics{./testwiz.jpg} \end{figure} \end{document} ############################## The testwiz.jpg was generated with ImageMagick from its built-in wizard: image:      convert wizard: -frame 1 testwiz.jpg      As it happens, this JPEG now has the following JFIF headers set, as seen in a hex editor: +-------------+-------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-------------------------+------------+------------+ | Value | 1 | 1 | 1 | +-------------+-------------------------+------------+------------+ | Meaning | "X/YDensities are DPI" | 1 dpi | 1 dpi | +-------------+-------------------------+------------+------------+ Exiftool reports this:     exiftool testwiz.jpg           Resolution Unit : inches      X Resolution : 1      Y Resolution : 1 Running lualatex mini.tex yields this:     LuaTeX warning: arithmetic: number too big     ! Dimension too large.      \dimen@                  l.7 \includegraphics{./testwiz.jpg}                                           ?      Hitting [Enter] repeatedly completes the PDF generation process. The resulting document has 66 kBytes. It shows only a white page in the viewer. Unpacking the PDF:     qpdf --qdf --object-streams=disable mini.pdf unpacked-mini.pdf      Inspecting unpacked-mini.pdf in a text editor shows these points: 1. The image data is embedded and complete. It is in PDF object no. 8, containing 65456 Bytes of JPEG data in the object's stream. 1. The /Contents object is no. 5 which contains this line of PDF source code:         -32645.5776 0 0 -32645.5776 124.802 706.129 cm /Im1 Do This means the image is displaced and scaled by a huge amount and is outside the normal page /MediaBox. After changing above line to             481 0 0 640 124.802 76.093 cm /Im1 Do              (be careful to ***overwrite*** the old bytes, and not add or subtract from the total number of bytes!) and saving the modification the image will become visible on the PDF page. I think it is stupid for lualatex to rely on the values of the JFIF header for scaling the image in (absence of explicit user provided TeX commands giving orders about the wanted scaling) when the headers says the resolution is 1 DPI: * Almost always a resolution of DPI = 1 does not make sense. * Almost always the resulting image on the page comes out in un-expected ways. Imagine a user with two identically looking files, where one file "works" on a PDF page and looks as expected while the other one exhibits the described behavior just because of one Bit being flipped in the JFIF header! You can manipulate the JFIF headers of a JPEG without changing any image pixels with the help of exiftool:     exiftool \       -jfif:Xresolution=300 \       -jfif:Yresolution=300 \       -jfif:ResolutionUnit=inches \        the.jpg Setting -jfif:ResolutionUnit=cm sets the "Units" field to "centimeters per inch". Setting -jfif:ResolutionUnit=None sets the "Units" field to "Xdensity and Ydensity are pixel ratios". The following settings in the JFIF header all currently cause different PDF output with the mini.tex input file: +-------------+-----------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-----------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-----------------------------+------------+------------+ | Value | 1 | 1 | 1 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 7 | 7 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 100 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 200 | 200 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 100 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 1 | 300 | 1 | +-------------+-----------------------------+------------+------------+ | Meaning | "X/YDensities are DPI" | px | px | +-------------+-----------------------------+------------+------------+ At least the behavior here is somehow consistent: the scaling, buckling and stretching happens according to the DPI settings. But if the settings are changed to these: +-------------+-----------------------------+------------+------------+ | Byte-Offset | 0x0d | 0x0e | 0x10 | +-------------+-----------------------------+------------+------------+ | Field-Name | "Units" | "XDensity" | "YDensity" | +-------------+-----------------------------+------------+------------+ | Value | 0 | 7 | 7 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 100 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 200 | 200 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 100 | 300 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 100 | +-------------+-----------------------------+------------+------------+ | Value | 0 | 300 | 1 | +-------------+-----------------------------+------------+------------+ | Meaning | "X/YDensities pixel ratios" | px | px | +-------------+-----------------------------+------------+------------+ then the output is identical for each case, not changing the pixel aspect ratios as it should. So... 1. lualatex does not respect the "X/Ydensity" JFIF fields if "Units" is set to "0", meaning "X/YDensities are pixel ratios". 2. lualatex does ONLY respect the "X/Ydensity" JFIF fields if "Units" is set to "1", meaning "X/YDensities are DPI". This is inconsistent. If it creates different output in case 2 it should also do so in case 1. I even think the lualatex PDF producing utility should NOT use the JFIF header metadata AT ALL for embedding JPEG images into pages. Because: * Most users are unaware of these JFIF headers. * Most users don't know how to change these headers. * These headers control the scaling of JPEGs on PDF pages only indirectly. * Most users expect to control the scaling of images in PDF output by different (LaTeX-provided) means directly. * The treatment of the various JFIF headers is inconsistent:     - if set as "X/Densities are DPI" they are NOT respected;     - if set as "X/YDensities are pixel ratios" they are (somehow) respected. # Executive summary 1. lualatex behaves unreasonable when it generates PDFs which include JPEG images:     - On the one hand, "XDensity" and "YDensity" fields are completely IGNORED if they describe "DPI" (or "DPcm").     - On the other hand, "XDensity" and "YDensity" fields ARE evaluated if they describ "pixel aspect ratios", but the outcome is totally unexpected for most users. 2. lualatex should ignore the JFIF fields completely and only obey the statements given by the document's LaTeX code. In the absence of such code, the PDF generators should assume 72 DPI. ---- This is the first time I report on Mantis. I don't know if my writeup and my tables will come out readable or not. If not, my apologies already now. Please also refer to these resources: 1. http://tex.stackexchange.com/q/244222/8747 2. http://tex.stackexchange.com/q/243753/8747 3. http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=27589 No tags attached.

#### Activities

 2015-05-11 18:32 reporter testwiz.jpg (65,456 bytes) testwiz.jpg (65,456 bytes) 2015-05-11 18:43 reporter   ~0001365 What a crappy user interface to use is this bug tracker here! For a willing and well-meaning user to have to deal with this is a total pain in the a*#. 1. Can't you at least use a mono-spaced font?! 2. And tell it not to eat up multiple spaces! 3. And give me a preview of what I've written before I finally submit? 4. And let me edit my posting after I discover a typo or an error?! 5. [... no, I'm not wasting my time any more...] As it stands, this will be used by me again... ...only in extreme emergencies! 2015-05-11 18:51 developer   ~0001366 Thank you for the report. We will investigate asap. Can you check against the latest 0.80.0 ? 2015-05-11 19:04 reporter   ~0001367 Luigi, I don't know how to check against 0.80.0. I totally rely on the packages the "Tex Live Utility" gives me (which I update almost daily). But if there isn't a *totally easy* and *easily revertible* way to update to 0.80.0 I'm not willing to jump through loops here. (I'm already handling other self-compiled software packages on my notebook. But these I *know* how to deal with in case of problems -- everything LaTeX/TeX related I have no clue. I only came to LaTeX/LuaLaTeX via pandoc...) 2015-05-11 19:18 developer   ~0001368 Confirmed with 0.80.1 (TeX Live 2015) (rev 5241) 2015-05-11 19:18 developer mini.tex (119 bytes) 2015-05-11 19:18 developer mini.log (6,615 bytes) 2015-05-11 20:34 developer   ~0001369 exiftools 9.46 with your image says : Resolution Unit : inches X Resolution : 1 Y Resolution : 1 : Image Size : 482x642 The image size is hence 482in x 642in, and both are bigger than 2^16pt (1pt =1/72.27 in) which is the max dimension accepted by the TeX engines pdftex, xetex, luatex. In this case the image is not loaded (in luatex) even if the user specifies a transformation matrix that gives an acceptable final dimensions. Note that 1dpi is not wrong by itself --- 20x20 at 1dpi is perfectly fine --- the limits are on the metric dimensions. Anyway, from luatex 0.80.0 there is a warning for 1dpi: LuaTeX warning: The image specifies an unusual resolution of 1dpi by 1dpi. LuaTeX warning: arithmetic: number too big LuaTeX warning: arithmetic: number too big 2015-05-12 00:59 reporter   ~0001370 @LuigiScarso: I know that the image size is 482x642 pixels, and a 1 DPI JFIF setting, if respected, will create an image of 482in x 642in. That is exactly my argument: scaling the image based on a (most likely) very completely uncontrolled JFIF header setting is not a good idea. The changed v0.80.0 warning does not help much, sorry. If the lualatex behavior does not change, it will still not create a visible image on the PDF page. 2015-05-12 02:21 developer   ~0001371 The image is valid, and as I have said there are 1dpi images that are perfectly fine with luatex. To put the box with the image on the main vertical list, Luatex needs to know its metric dimensions and the only way, when the dpi is known, is to multiply the dpi for the width and the height. In this case the dimensions are over 2^16pt, and the box cannot be built. It is supposed that a user knows these features, checking if an image fits into this limit, and acting as consequence if this is not the case. We agree that it's desirable that a transformation matrix could act before the calculation of the dimensions, but this doesn't solve the problem, as also impose a scaling to 2^16pt . This is not a bug, but a feature that could be improved in a next release. 2015-05-12 08:40 developer   ~0001372 Ah, sorry, I meant 32768pt = 2^15pt not 2^16pt. 2015-05-12 09:07 developer   ~0001373 Also the message LuaTeX warning: arithmetic: number too big comes from scaled ext_xn_over_d(scaled x, scaled n, scaled d) {     double r = (((double) x) * ((double) n)) / ((double) d);     if (r > DBL_EPSILON)         r += 0.5;     else         r -= 0.5;     if (r >= (double) max_integer || r <= -(double) max_integer)         luatex_warn("arithmetic: number too big");     return (scaled) r; } where #define max_integer 0x7FFFFFFF so it's related to a value that cannot fit into a scaled (which is defined as int). Even if the image fits into a scaled, it can be still problematic because the \maxdimen is 16383.99998pt ~= 2^14pt . 2015-10-23 09:07 manager   ~0001473 it's dangerous to change behaviour of the inclusion and interpretation of fields without much testing, we keep an eye on it but zero or one pixel images are ot common in tex related workflows 2015-11-02 17:40 manager   ~0001495 we will not touch the inclusion code; this issue relates to the maximum dimension that tex can handle and 1 dpi is resulting in an overflow ... so be it (LS/HH)

#### Issue History

2015-05-11 18:32 pdfkungfoo New Issue
2015-05-11 18:32 pdfkungfoo File Added: testwiz.jpg
2015-05-11 18:43 pdfkungfoo Note Added: 0001365
2015-05-11 18:51 luigi scarso Note Added: 0001366
2015-05-11 19:04 pdfkungfoo Note Added: 0001367
2015-05-11 19:18 oneiros Note Added: 0001368
2015-05-11 19:18 oneiros File Added: mini.tex
2015-05-11 19:18 oneiros File Added: mini.log
2015-05-11 19:20 oneiros Product Version 0.79.1 => 0.80.1
2015-05-11 20:34 luigi scarso Note Added: 0001369
2015-05-12 00:59 pdfkungfoo Note Added: 0001370
2015-05-12 02:21 luigi scarso Note Added: 0001371
2015-05-12 08:40 luigi scarso Note Added: 0001372
2015-05-12 09:07 luigi scarso Note Added: 0001373
2015-10-23 09:07 Hans Hagen Note Added: 0001473
2015-10-23 09:10 Hans Hagen Target Version => maybe never (> v 1)
2015-10-24 21:01 Hans Hagen Status new => acknowledged
2015-11-02 17:40 Hans Hagen Note Added: 0001495
2015-11-02 17:40 Hans Hagen Status acknowledged => closed
2015-11-02 17:40 Hans Hagen Assigned To => Hans Hagen
2015-11-02 17:40 Hans Hagen Resolution open => won't fix