nanogui: Thread: Segfault in fblin32.c, line 77

Hi,

My Nano-X-binary under Xorg in Ubuntu amd64 on a 64-bit AMD Athlon x2 
produces the following segfault on start-up:

Output of gdb:

#0  0x0000000000425ae5 in linear32_drawhorzline (psd=0x6df4c0, x1=511, 
x2=639, y=240, c=0) at src/drivers/fblin32.c:77
#1  0x000000000041834a in gen_fillrect (psd=0x6df4c0, x1=0, y1=241, 
x2=639, y2=479, c=0) at src/drivers/genmem.c:80
#2  0x000000000041a25f in X11_fillrect (psd=0x6cc920, x1=0, y1=0, 
x2=639, y2=479, c=0) at src/drivers/scr_x11.c:979
#3  0x0000000000404954 in GdOpenScreen () at src/engine/devopen.c:161
#4  0x000000000042c9b0 in GsInitialize () at src/nanox/srvmain.c:1073
#5  0x000000000042c1c9 in main (argc=1, argv=0x7fff2d897f08) at 
src/nanox/srvmain.c:210

All calls to linear32_drawhorzline(..) from within gen_fillrect() are 
returning without problems, as long as y1 is smaller than 240.

Some ideas??

Best regards
Pascal

: #0  0x0000000000425ae5 in linear32_drawhorzline (psd=0x6df4c0, x1=511, 
: x2=639, y=240, c=0) at src/drivers/fblin32.c:77

One possibility is that the framebuffer is reporting an invalid size
to the startup in drivers/scr_fb.c, which results in the initial
screen clear out-of-bounds at y=240 (rather than, say 239)

Regards,

Greg

: #0  0x0000000000425ae5 in linear32_drawhorzline (psd=0x6df4c0, x1=511, 
: x2=639, y=240, c=0) at src/drivers/fblin32.c:77
: #1  0x000000000041834a in gen_fillrect (psd=0x6df4c0, x1=0, y1=241, 
: x2=639, y2=479, c=0) at src/drivers/genmem.c:80
: #2  0x000000000041a25f in X11_fillrect (psd=0x6cc920, x1=0, y1=0, 
: x2=639, y2=479, c=0) at src/drivers/scr_x11.c:979

Oops I just noticed that you're running nano-X on X11, not FB.

The X11 driver keeps "savebits" by commanding X11, then
drawing using the fb32 driver.  It looks like perhaps the
size malloced could be incorrect.  This is calced in the genmem.c
file, I think, but you'll need to look hard at the X11 driver to
trace where it alloced the savebits fb buffer in the first place.

I'm not aware of any X11 driver problems, so its a bit strange...

Regards,

Greg

Hi Greg

Greg Haerr schrieb:

[...]

> The X11 driver keeps "savebits" by commanding X11, then
> drawing using the fb32 driver.  It looks like perhaps the
> size malloced could be incorrect.  This is calced in the genmem.c
> file, I think, but you'll need to look hard at the X11 driver to
> trace where it alloced the savebits fb buffer in the first place.
> 
> I'm not aware of any X11 driver problems, so its a bit strange...
> 
> Regards,
> 
> Greg

I found the following:

The size malloced in scr_x11.c is calculated by GdCalcMemGCAlloc(..) 
which returns 1228800 (= 640*480*4) for TrueColor (32bit).
savebits.addr is of type ADDR32 which is a typedef to a long*. On my 
64bit system, a long is 8 bytes, so the calculation

"addr += x1 + y * psd->linelen;"

in fblin32.c adds x1+y*psd->linelen*8 bytes (instead of 4 bytes, which 
were used for calculating the size for mallocing). That's the reason why 
I have the segfault when calculating with y=241.

I changed in drivers/fb.h the typedefs for ADDR8, ADDR16 and ADDR32 to 
the standard sized types u_int8_t, u_int16_t and u_int32_t.

Thanks for the input!

Best regards
Pascal