nanogui: Thread: Images not drawn (FLTK+nxlib app with nano-X)

This is a new thread which I believe is related to 'Font usage/setup' problem described by me previously.

The scenario is the same, I am running a FLTK app linked with nxlib with nano-X server.
The app works CORRECTLY with X11 server on the same machine.
The problem is that most of my images (mostly 32x32 icons) are not drawn/shown at all.
Some images are being loaded from files but most are statically stored in arrays (FLTK loads them and puts them into code during code generation).
There is one image that shows up correctly and one that is almost right (there is some color issue that is not exactly right but I will get to that another time).

When I checked the foreground/background values, they were both zero which I already discovered do not work properly when drawing fonts.
After I changed the foreground color to 1, there was no change; still no images.
This makes sense to me because the foreground/background values should not be used when drawing images because it should be a straight memory copy from an image array to frame buffer (with some conversions).

My belief is that the images are either drawn transparent or not drawn at all but what would decide why some images are drawn and some are not?

I will get deeper into the drawing code of nxlib/nano-X to solve this but before I do that I would like to ask:
Does anybody has any suggestions or experiences with image drawing with nxlib/nano-X?

By the way, the rest of the app draws everything else (not images) correctly, but I think it uses lines and points to draw.

Martin

> The problem is that most of my images (mostly 32x32 icons) are not
drawn/shown at all.
Some images are being loaded from files but most are statically stored in
arrays (FLTK loads them and puts them into code during code generation).
There is one image that shows up correctly and one that is almost right
(there is some color issue that is not exactly right but I will get to that
another time).

Image display using NXLIB can run into problems quickly.  Xlib has a HUGE
section of code for XPutImage that will convert any format image to any
other format, negotiated with the XServer, etc.  Since I had no desire to
rewrite all of X11's image code, we don't perform all these conversions.
That is to say, X11 will allow any bpp image format to be displayed,
regardless
of the server hw format.   With nano-X, originally, we only allowed a single
bitmap format (not to be confused with MWIMAGEHDR format, which is
different, here we're talking about pixmap formats): we allowed only the
hw bpp format.  That is, if you're running 16bpp 565, then the pixmap
had to be in that format.  In most cases, this was never an issue, including
almost all X11 programs I've come across.  Except - we needed to handle
1bpp (monochrome) images, regardless of hw bpp format.  So there
is some code in NXLIB to handle that conversion; although it isn't complete,
since nano-X itself doesn't actually handle that format (which is the
ultimate
proper fix).

I don't know yet what the issue is with your images, but we need to get some
debug code that shows what format the image is arriving in (I think there's
already debug code in Image.c::putImage() that does this).

Please give some more detailed info so I can help debug this issue.



> When I checked the foreground/background values, they were both zero which
I already discovered do not work properly when drawing fonts.
After I changed the foreground color to 1, there was no change; still no
images.
This makes sense to me because the foreground/background values should not
be used when drawing images because it should be a straight memory copy from
an image array to frame buffer (with some conversions).

Agreed.  Image output doesn't use fg/bg values, unless its a monochrome
image,
see above.


> My belief is that the images are either drawn transparent or not drawn at
all but what would decide why some images are drawn and some are not?

I'm not aware of Xlib supporting transparency on pixmaps without extensions.
We
need to find what mechanism is being used for transparent drawing.  Mono
bitmaps
can be drawn using fg only, thus allowing single color transparent drawing.

Regards,

Greg

Greg,
I fixed it (almost) by making following changes:

--- SetClip.c.org	2003-05-18 04:06:08.000000000 +0000
+++ SetClip.c	2006-01-26 11:58:00.642885864 +0000
@@ -4,7 +4,12 @@
 int
 XSetClipOrigin(Display *display, GC gc, int clip_x_origin, int clip_y_origin)
 {
+	GR_GC_INFO	gcip;
+
 	GrSetGCClipOrigin(gc->gid, clip_x_origin, clip_y_origin);
+
+	GrGetGCInfo(gc->gid, &gcip);
+	GrOffsetRegion(gcip.region, clip_x_origin, clip_y_origin);
 	return 1;
 }
 
I would like to know if it makes sense to you and why GrSetGCClipOrigin() did not do it correctly?
Why the call to GrOffsetRegion() is necessary?
If it makes sense to you, then you can check this into nxlib CVS.

The only problem now is that some of my images are slightly clipped on the right edge. I think this is a rounding to the byte/word(?) boundary.
Is there a problem with image widths not divisible by 8?

Martin


-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Tuesday, January 24, 2006 4:18 PM
To: Martin Kajdas
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


Martin -
    Thanks for the info, seems like you're on to the one or
more problems.  I'm slammed with work and don't have the
source handy, and leaving for a road trip tomorrow.  So here's
some thoughts to keep you going:

To find the clipping code in NXLIB, grep for Clip should
find it, I can't recall having to set clipping for images directly,
but FLTK could easily be doing it.  Just commenting out the
user clip call would be a good start.  IIRC, FLTK has issues
with hard-locations for windows, and perhaps there needs to
be an offset passed (or clip offset API called) in order to
fix the clip issue.

I can't remember where we check the transparent color of an icon,
and I think it might be a kluge where the 0,0 location is used for
the transparent color.  Again, comment it out, perhaps we need
a better algorithm for determining the transparent color (like inspecting
the mask?)  Also - the hard coordinates issue with FLTK could
still be an issue here; when we wrote the PIXIL PDA solution,
we had problems when we moved the applicaiton around,
FLTK stored coords differently than nano-X did.

Hopefully this helps, I'll be around later tonight also.

Regards,

Greg

What I wrote in the previous email is 'almost' correct.
What I noticed also, is that the icon's transparent color of the icon is
also referenced at 0,0.
When the icon is drawn at 10,10 the image is drawn at 10,10 but it is
overlaid with transparency layer referenced at 0,0.
Therefore, it may not be due to only clipping but to image overlaid with
transparency layer at 0,0 (wrong!) location.

This also explains my other problem with the icon being 'wrong' color as
mentioned in my original email.
At that time I did not think the two problems were related.

One other hint, I have one image being draw correctly even though it is not
at 0,0 but at the 0,0 location of a new window.
This makes me believe that it is not related to absolute location of the
screen but of the currently active window.

Martin


-----Original Message-----
From: Martin Kajdas
Sent: Tuesday, January 24, 2006 3:18 PM
To: 'Greg Haerr'
Subject: RE: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


Greg,

I have a very good hint and with it I believe we can fix this.
The problem is that the clipping region is referenced to the 0,0 location of
the screen and not the location of the image.
I tested it and I can see the second icon if I can place it close to the
upper left corner.

To make it clear here is an example:
If I place the icon at 0,0 location of the screen, I can see the whole icon.
If I place the icon at 10,10 location of the screen, I can see the icon with
10 pixels of it shaved from right and bottom of the image.

Because my icons are small and way away from 0,0 location they are not
showing because the are outside clipping region located at 0,0.

Let me know where in code (nxlib I believe) we need to fix this.
Martin


-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Thursday, January 19, 2006 9:55 PM
To: Martin Kajdas
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


Thanks Martin.  I've got to leave town for a couple of days, hopefully
I can get to this on Sunday.  I'll continue to help you get this solved...

Regards,

Greg


I reduced my code to minimum to show the problem.
The code 'icontest.fl' is FLTK-1.1.6 (fluid) code which then generates
'icontest.h' and 'icontest.cxx'.
I statically link it with nxlib-0.45 and other required libraries and
generate 'icontest' linux executable.
I am using framebuffer display with 5:6:5 RGB color resolution.
I run it with nano-X as usual and the result is that one icon is displayed
(MKlogoIcon.png) and the other (filenew.png) is not. The FLTK loads the png
files and stores them in code but I also tried to load them dynamically at
run time with the same results.

I provided you all of the files except FLTK.
Try to run 'icontest' with nano-X and see if you get my results.
If so, try to use my icons with your program to see if the problem is
somehow related to my icons (size, color,...)
Both icons show up correctly with X11 server, so the problem is with nxlib
or nano-X.

Martin




-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Wednesday, January 18, 2006 12:43 PM
To: Martin Kajdas; ####@####.####
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


> The problem is that most of my images (mostly 32x32 icons) are not
drawn/shown at all.
Some images are being loaded from files but most are statically stored in
arrays (FLTK loads them and puts them into code during code generation).
There is one image that shows up correctly and one that is almost right
(there is some color issue that is not exactly right but I will get to that
another time).

Image display using NXLIB can run into problems quickly.  Xlib has a HUGE
section of code for XPutImage that will convert any format image to any
other format, negotiated with the XServer, etc.  Since I had no desire to
rewrite all of X11's image code, we don't perform all these conversions.
That is to say, X11 will allow any bpp image format to be displayed,
regardless
of the server hw format.   With nano-X, originally, we only allowed a single
bitmap format (not to be confused with MWIMAGEHDR format, which is
different, here we're talking about pixmap formats): we allowed only the
hw bpp format.  That is, if you're running 16bpp 565, then the pixmap
had to be in that format.  In most cases, this was never an issue, including
almost all X11 programs I've come across.  Except - we needed to handle
1bpp (monochrome) images, regardless of hw bpp format.  So there
is some code in NXLIB to handle that conversion; although it isn't complete,
since nano-X itself doesn't actually handle that format (which is the
ultimate
proper fix).

I don't know yet what the issue is with your images, but we need to get some
debug code that shows what format the image is arriving in (I think there's
already debug code in Image.c::putImage() that does this).

Please give some more detailed info so I can help debug this issue.



> When I checked the foreground/background values, they were both zero which
I already discovered do not work properly when drawing fonts.
After I changed the foreground color to 1, there was no change; still no
images.
This makes sense to me because the foreground/background values should not
be used when drawing images because it should be a straight memory copy from
an image array to frame buffer (with some conversions).

Agreed.  Image output doesn't use fg/bg values, unless its a monochrome
image,
see above.


> My belief is that the images are either drawn transparent or not drawn at
all but what would decide why some images are drawn and some are not?

I'm not aware of Xlib supporting transparency on pixmaps without extensions.
We
need to find what mechanism is being used for transparent drawing.  Mono
bitmaps
can be drawn using fg only, thus allowing single color transparent drawing.

Regards,

Greg

> I would like to know if it makes sense to you and why GrSetGCClipOrigin()
did not do it correctly?
> Why the call to GrOffsetRegion() is necessary?

Martin -
    I don't understand this yet either.  Before your fix, the region is
offset
in the GsPrepareWindow code, which ultimately calls GdOffsetRegion.
Your fix sets the region offset immediately...


> If it makes sense to you, then you can check this into nxlib CVS.

Yes, I'll add it.



> The only problem now is that some of my images are slightly clipped on the
right edge. I think this is a rounding to the byte/word(?) boundary.
> Is there a problem with image widths not divisible by 8?

Yes - the MWIMAGEBITS typedef is unsigned short, which requires
images to be a multiple of 16 (two bytes).  It seems this could be a problem
with X11 images being allowed with no short-padded boundary.  However,
this should cause the images to be one dot too large, not clipped.  Please
comment.

Regards,

Greg

Regarding 'the MWIMAGEBITS typedef is unsigned short'
I verified that the images with widths multiple of 16 pixels are drawn correctly.
Others are trimmed to smaller size (multiple of 16).

Looks like the image width is rounded down to multiple of 16 instead of up.
Maybe the rounding problem is in the nxlib before it calls microwindows.
Martin



-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Tuesday, January 31, 2006 7:33 AM
To: Martin Kajdas
Cc: ####@####.####
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


> I would like to know if it makes sense to you and why GrSetGCClipOrigin()
did not do it correctly?
> Why the call to GrOffsetRegion() is necessary?

Martin -
    I don't understand this yet either.  Before your fix, the region is
offset
in the GsPrepareWindow code, which ultimately calls GdOffsetRegion.
Your fix sets the region offset immediately...


> If it makes sense to you, then you can check this into nxlib CVS.

Yes, I'll add it.



> The only problem now is that some of my images are slightly clipped on the
right edge. I think this is a rounding to the byte/word(?) boundary.
> Is there a problem with image widths not divisible by 8?

Yes - the MWIMAGEBITS typedef is unsigned short, which requires
images to be a multiple of 16 (two bytes).  It seems this could be a problem
with X11 images being allowed with no short-padded boundary.  However,
this should cause the images to be one dot too large, not clipped.  Please
comment.

Regards,

Greg

> Looks like the image width is rounded down to multiple of 16 instead of
up.

yes, agreed


> Maybe the rounding problem is in the nxlib before it calls microwindows.

I'm pretty sure we don't have an image boundary size bug in
microwindows.  A quick look into the nxlib Image.c could
help.  Since microwindows always expects word-padding, there
could be an issue with reading a non-existent byte
as the last pixel of an image line.  The chop code could have
been placed there for this protection.

Regards,

Greg

I put many print messages into Image.c (putTrueColorImage()) to debug this and I verified that the image size and the data is correct when calling GrArea() function (width=height=24, 48x24 bytes of data match image).
I used a 24x24 image which displays only 16x24 worth of data.

From this, I conclude that the nxlib is OK and the problem is in nano-X.
I am using 5:6:5 color format with framebuffer.

Where should I be looking now? GdArea()?

One problem I noticed with nxlib (PNG conversion), is that the transparent color and black color are both zero making the images appear wrong (black pixels are transparent).
This would explain another problem with my images.
Martin


-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Wednesday, February 01, 2006 3:55 PM
To: Martin Kajdas
Cc: ####@####.####
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


> Looks like the image width is rounded down to multiple of 16 instead of
up.

yes, agreed


> Maybe the rounding problem is in the nxlib before it calls microwindows.

I'm pretty sure we don't have an image boundary size bug in
microwindows.  A quick look into the nxlib Image.c could
help.  Since microwindows always expects word-padding, there
could be an issue with reading a non-existent byte
as the last pixel of an image line.  The chop code could have
been placed there for this protection.

Regards,

Greg

: I verified that the image size and the data is correct when calling
GrArea() function (width=height=24, 48x24 bytes of data match image).
: I used a 24x24 image which displays only 16x24 worth of data.

I thought the problem occured with odd widths, not even widths...
Can you explain a bit more?



: Where should I be looking now? GdArea()?

Yes.  There is a possibility that the nxproto.c code could be
chopping/rounding the data down, although I don't think so.
This would be the GrArea code in srvnet.c/client.c.



: One problem I noticed with nxlib (PNG conversion), is that the transparent
color and black color are both zero making the images appear wrong (black
pixels are transparent).
: This would explain another problem with my images.

The image_png.c file should have the transparent member in the
MWIMAGEHDR structure set to -1.  You might verify this to
be the case.

Regards,

Greg

The problem is with image widths not equal to 16 bit multiples,
i.e. the image of size 32x24 displays fine,
the image of size 24x24 displays only 16x24 pixels (the 8 right most pixels are either transparent or not displayed)
I believe, any width between 17 and 31 will display only 16 pixels.
It is not odd/even issue but 16 bit multiple (works) vs. not 16 bit multiple (clipped).

I will look at your other suggestions tomorrow morning.
Martin


-----Original Message-----
From: Greg Haerr ####@####.####
Sent: Thursday, February 02, 2006 2:57 PM
To: Martin Kajdas
Cc: ####@####.####
Subject: Re: [nanogui] Images not drawn (FLTK+nxlib app with nano-X)


: I verified that the image size and the data is correct when calling
GrArea() function (width=height=24, 48x24 bytes of data match image).
: I used a 24x24 image which displays only 16x24 worth of data.

I thought the problem occured with odd widths, not even widths...
Can you explain a bit more?



: Where should I be looking now? GdArea()?

Yes.  There is a possibility that the nxproto.c code could be
chopping/rounding the data down, although I don't think so.
This would be the GrArea code in srvnet.c/client.c.



: One problem I noticed with nxlib (PNG conversion), is that the transparent
color and black color are both zero making the images appear wrong (black
pixels are transparent).
: This would explain another problem with my images.

The image_png.c file should have the transparent member in the
MWIMAGEHDR structure set to -1.  You might verify this to
be the case.

Regards,

Greg

: I verified that the image size and the data is correct when calling
GrArea() function (width=height=24, 48x24 bytes of data match image).

I'm still confused here.  With width 24, padded to 16 bits will
mean we'd need 32 bits, not 48 as above.  Going with
height 1 for simplicity, a 24x1 image would require 4x1=4
bytes to be transferred to the server.

Regards,

Greg