Fix crash while scaling large bitmaps.
Fast bitmap scaling overflowed the LUT used by the nearest-neighbor algorithm.
When a bitmap has 46k pixel on a side and is enlarged, the scaling code
overflows the 32-bit long, resulting in negative indexes, which then segfaults.
This isn't as rare as it sounds. At least in web-view in writer the border/shadow
bitmap is as long as the document (which is an issue in its own right,)
which can overflow for large documents during scaling and segfault.
Change-Id: I1ccf73d02469f6601a9a7e67b30524cb497cf6bc
Reviewed-on: https://gerrit.libreoffice.org/14597
Reviewed-by: Caolán McNamara <caolanm@redhat.com>
Tested-by: Caolán McNamara <caolanm@redhat.com>
diff --git a/vcl/source/gdi/bitmap3.cxx b/vcl/source/gdi/bitmap3.cxx
index 2bd586f..bfd25a6 100644
--- a/vcl/source/gdi/bitmap3.cxx
+++ b/vcl/source/gdi/bitmap3.cxx
@@ -1075,18 +1075,19 @@ bool Bitmap::ImplScaleFast( const double& rScaleX, const double& rScaleY )
const long nScanlineSize = pWriteAcc->GetScanlineSize();
const long nNewWidth1 = nNewWidth - 1L;
const long nNewHeight1 = nNewHeight - 1L;
const long nWidth = pReadAcc->Width();
const long nHeight = pReadAcc->Height();
boost::scoped_array<long> pLutX(new long[ nNewWidth ]);
boost::scoped_array<long> pLutY(new long[ nNewHeight ]);
if( nNewWidth1 && nNewHeight1 )
{
const double nWidth = pReadAcc->Width();
const double nHeight = pReadAcc->Height();
boost::scoped_array<long> pLutX(new long[ nNewWidth ]);
boost::scoped_array<long> pLutY(new long[ nNewHeight ]);
for( long nX = 0L; nX < nNewWidth; nX++ )
pLutX[ nX ] = nX * nWidth / nNewWidth;
pLutX[ nX ] = long(nX * nWidth / nNewWidth);
for( long nY = 0L; nY < nNewHeight; nY++ )
pLutY[ nY ] = nY * nHeight / nNewHeight;
pLutY[ nY ] = long(nY * nHeight / nNewHeight);
long nActY = 0L;
while( nActY < nNewHeight )
