nanogui: Thread: Conversion of PIXELVALs <-> COLORVALs

Morten,
    I sat down and wrote the conversion macros pertaining to our discussion
about having client capabilities to convert between Microwindows
COLORVALs and PIXELVALs.  These macros allow a client program
to specify in hardware format values, or portable RGB values, and translate
between them.  Note that these are intended for 8/8/8, 5/6/5, and 3/3/2
truecolor systems only; palettized systems will require function calls
that compare system palette entries before using GrArea().  This will
allow fast image processing using either the updated GrArea() call that
works with 8, 16 or 24 bpp PIXELVALs, or the new GrAreaRGB()
call that takes COLORVALs as image bits.

Have fun.

/*
 * Conversion from RGB to PIXELVAL
 */
/* create 24 bit 8/8/8 format pixel (0x00BBGGRR) from RGB triplet*/
#define RGB2PIXEL24(r,g,b) \
 (((b) << 16) | ((g) << 8) | (r))

/* create 16 bit 5/6/5 format pixel from RGB triplet */
#define RGB2PIXEL565(r,g,b) \
 ((((r) & 0xf8) << 8) | (((g) & 0xfc) << 3) | (((b) & 0xf8) >> 3))

/* create 8 bit 3/3/2 format pixel from RGB triplet*/
#define RGB2PIXEL332(r,g,b) \
 (((r) & 0xe0) | (((g) & 0xe0) >> 3) | (((b) & 0xc0) >> 6))

/*
 * Conversion from COLORVAL to PIXELVAL
 */
/* create 24 bit 8/8/8 format pixel (0x00BBGGRR) from RGB colorval(0x00BBGGRR)*/
#define COLOR2PIXEL24(c) \
 ((c) & 0x00ffffff)

/* create 16 bit 5/6/5 format pixel (0x00BBGGRR) from RGB colorval(0x00BBGGRR)*/
#define COLOR2PIXEL565(c) \
 ((((c) & 0xf8) << 8) | (((c) & 0xfc00) >> 5) | (((c) & 0xf80000) >> 19))

/* create 8 bit 3/3/2 format pixel (0x00BBGGRR) from RGB colorval(0x00BBGGRR)*/
#define COLOR2PIXEL332(c) \
 (((c) & 0xe0) | (((c) & 0xe000) >> 11) | (((c) & 0xc00000) >> 22))

/*
 * Conversion from PIXELVAL to red, green or blue components
 */
/* return 8 bit r, g or b component of 24 bit (0x00BBGGRR) pixelval*/
#define PIXEL24RED(pixelval)  ((pixelval) & 0xff)
#define PIXEL24GREEN(pixelval)  (((pixelval) >> 8) & 0xff)
#define PIXEL24BLUE(pixelval)  (((pixelval) >> 16) & 0xff)

/* return 5/6/5 bit r, g or b component of 16 bit pixelval*/
#define PIXEL565RED(pixelval)  (((pixelval) >> 11) & 0x1f)
#define PIXEL565GREEN(pixelval)  (((pixelval) >> 5) & 0x3f)
#define PIXEL565BLUE(pixelval)  ((pixelva) & 0x1f)

/* return 3/3/2 bit r, g or b component of 8 bit pixelval*/
#define PIXEL332RED(pixelval)  (((pixelval) >> 5) & 0x07)
#define PIXEL332GREEN(pixelval)  (((pixelval) >> 2) & 0x07)
#define PIXEL332BLUE(pixelval)  ((pixelval) & 0x03)

Greg

Greg Haerr wrote:
> 
> Morten,
>     I sat down and wrote the conversion macros pertaining to our discussion
> about having client capabilities to convert between Microwindows
> COLORVALs and PIXELVALs.

Sorry for being on-and-off, but you are right; busy times.
I'll try to comment on all your previous mails here (it is long...):

Please note:  Some of my problem may be that I don't understand
completely (or at least don't realize the consequences) of the
multiple personalities of Microwindows.  I'm primarily interested
in Nano-X client/server, and may not quite grasp all the issues
WRT running without a Nano-X server for instance.

> Proposed solution:
>     o add GrAreaRGB() that takes COLORVALs rather than PIXELVALs,
> and have the server completely translate the passed RGB values per pixel
> to the current truecolor or palette colors.  This allows device-independent
> GrArea to be called for minimal client computation, when speed isn't entirely
> of the essence.

This is a very good idea, and an easy way out for small undemanding apps.
More on this later.

> RGB2PIXEL(r,g,b)    returns PIXELVAL according to compiletime or runtime PF_XXX?
>     It's possible that we might need this to operate at runtime, rather than
> compiletime.

IMO the runtime version is only needed when there is a chance that the
application may be run on a server that doesn't support the mode used
when compiling the application.  This is something we may want to avoid,
but when it can't be avoided, the server can translate if available and
enabled.  The server has the benefit of being able to allocate translation
tables only once, and not once for each client which would be the case for
client side conversion.

> For the simpler compiletime case, we can only make it work when running in the
> truecolor
> modes:
>     PF_TRUECOLOR24            (32bpp)
>     PF_TRUECOLOR565          (16bpp)
>     PF_TRUECOLOR332          (8bpp)
> For PF_PALETTE, we're kinda in trouble, unless we're running a fixed palette.

Yes, or use a function, possibly inline, to translate at client for PALETTE
mode.
Translating at the server gives the "central translation tables" benefit, but
at the expense of more data transferred to the server.

> So to solve this at compiletime (the fastest), I suggest the following macros:
> 
> RGB2PIXEL24(r,g,b)        convert RGB to 32bpp 24bit truecolor
> RGB2PIXEL565(r,g.b)      convert RGB to 16bpp 5/6/5 truecolor
> RGB2PIXEL332(r,g,b)      convert RGB to 8bpp 3/3/2 truecolor

In my original proposal, you don't specify which mode you want to
convert to, it is implied by the NANOX_PF macro.  It should probably
be possible to redefine the NANOX_PF macro and re-include the
header file to change to another pixel format in case the application
wants to support several different pixel formats, like an MPEG
player that wants to be as efficient as possible for several pixel
formats.  Or one could just put the different code in different
files with their own NANOX_PF pixel format selection.

> For runtime conversion, a single RGBToPixel() routine could run on the client
> side and statically get the PF_XXX pixtype thru GrGetScreenInfo once, and then
> calculate a PIXELVAL from the passed parameters.

I don't think we need to support this, as I don't think anyone
will use it due to performance problems (When using the Area
function, you do it for the exact reason that you *don't* want
to handle pixels individually by a "slow function").

> It could work in PF_PALETTE
> mode by calling GrGetSystemPaletteEntries.  This is slower, but can work
> regardless
> of the server's PIXELVAL size.  (But probably isn't needed by you)

Yes, and the usage could be coherent with the true color cases:

   RGB2PIXELPAL(r,g,b)

using your naming scheme. This would be kind of a hybrid solution, but
I think this could work.  Make the conversion in the client, and cache
the palette. This would require notification of palette changes for
cache flushing.

> For reading PIXELVALs, the following macros could also be needed, again
> depending on the type of PIXELVAL:
> 
> 
> PIXEL24GREEN(pixelval)    returns the 8 bit green component of 24bpp
>                           pixelval
> PIXEL565GREEN(pixelval)   returns the 6 bit green component of
>                           16bpp pixelval
> PIXEL332GREEN(pixelval)   returns the 3 bit green component of
>                           8bpp pixelval
> 
> and likewise for RED and BLUE.

I will again prefer not to have each macro named after its pixel type,
as this only complicates the application code IMHO.  If we aim to support
subtly and not so subtly different pixel formats, it would be nice if
these macros that are supposed to hide the differences don't differ
in name!

> With the above design, a Nano-X applications programmer has two choices in which
> to write his application:  (1) assume knowledge of the PIXELVAL implementation,
> in which case he uses the macros, and the application only runs on that
> implementation, or he can use the runtime functions, which are slower, and
> the application will run on any implementation.

The runtime version is something that could be supported in order to
provide a guaranteed working application, but the question is if
anyone will use it if it turns out to be slow.

I much more favor the idea of having the server translate if the need
arises, and guarantee that a certain 24 bit true color format will
always be supported in the server (possibly by translation).  This
will be both portable and provide the best quality (at a memory and
performance penalty).

> : Yes, this is true. But I don't like the concept of *assuming* that
> : pixles are 16 bits - if they are not, we run into trouble.  The
> : reasoning behind my design was; make it possible to make it possible
> : (later) to write apps and have them run everywhere, if the Nano-X
> : server is compiled with the proper settings, without modifying
> : the clients.
> 
> That was the whole idea behind using RGB on clients.  The clients
> are compiled once, and run on any server, truecolor and palettized.

This is a special case of what I propose; Guarantee that the server
will translate the 24bpp pixel format correctly onto whatever screen
used, everywhere.

For a lot of programs this is fine, for some that needs every available
cycle or memory it is a little bit more problematic, so we provide a
way to create 8 or 16 bit pixels *fast* and ship them over.

For autonomous systems, this is just great.  If we later find that
we need to support 16 bit only apps on 24 bit color etc., we have the
option of letting the server translate.

> The apps are written in a standard color model and will guarantee
> be runnable on any server.

Giving the users a choice of using the format always available if
porting is a concern, and hit the iron "directly" if speed/space
is important, would be very nice.  If the user wants to support both
formats (or even more) in the same application, he is free to do so.

> What you are saying is that you want an additional, non-RGB
> format, that also is client-portable, but is closer to the hardware,
> so that it's faster.

Yes, but: It is *optionally* portable; if the application has special
requirements, they must be met by the server either directly or
through translation.  If you want a small server and small apps,
you can't mix pixel formats.  If you want to be able to mix
certain formats with minimum speed penalty, you can support
more formats in the application or translate in the server.
Both require more memory to work.

> Because of this, the client has to include
> alot more code, since it needs to be able to run on palettized
> systems also, not just fast 16bpp truecolor.

16bpp -> 8bpp can be done in the server as well, and quickly
too if you have 64K of memory that can be used for a table
lookup.  It will not be floyd-steinberg quality or as fast/small
as possible, but it will work.

> I think that
> having a mechanism to allow a client to know about the
> hardware color format and run quickly is a good idea,
> but having two supposedly client-portable formats is
> probably a bad idea, considering this is supposed to be a small
> project.

Greg, nothing stops anyone from keeping it a small project if we
are talking about the size of the object code.  Just disable all
the stuff you don't need (like translation) and use the 24bpp
format if portability is important.  I will not need translation
and probably not write any, but I *do* need the rawpixel format
handling for speed and memory concerns, and when implementing
that - I want to do it right; I try to make it extensible and
flexible.  Another thing with splitting things up like this is
that it possibly makes distributed development easier as we will
not step on each other toes so often (when I try to optimize the
16bpp path fx.)

If you are worried about the total size of the source code and the
quality of the various parts of the system, I agree with you -
but performance is still very important, along with small object
code size which is even more important.

> However, I don't think this is the whole problem.  The
> real issue is that GrArea() only takes PIXELVALs, and I 
> don't like the idea that portable programs should have to know
> about the hardware color format.  (yes, in your case, they might
> want to, but don't have to).

The "don't have to" should be a guide, not a limitation IMO.  The
documentation should clearly state which methods are portable and
which are not, and when they should and could be used.

> I propose that the first solution be that GrAreaXXX() be allowed
> to be passed COLORVALs, so that small images, and most
> drawing be allowed to be standardized, and passed as RGB.
> In this way, any image can be displayed in a standard
> format from Nano-X on any server.

Agreed, but they don't have to be COLORVALs.  Just state that
the pixel format PF_TRUECOLOR888 will always be supported by
the server, and can thus always be used (by using the default
pixel format).  The RGB2PIXEL macros will do the right thing
in this case, and always work.

> In addition, we implement 3 more GrAreaXXX's, one for 8, 16, 
> and 32bpp PIXELVAL sizes.  These additional
> opcodes can be used, _only_ if the SCREENINFO indicates
> that the hardware supports it.

Agreed, but the SCREENINFO should indicate what the hardware
supports, and what the *server* supports.  Obviously, an
application that can do multiple pixel formats uses the
one the hardware uses if available, else one that the
server supports.  With translation, the server can support
more formats than the single one provided by the hardware.

> [I still don't see the reasoning
> behind the need for multiple 16bpp GrArea's.  Since the
> server doesn't modify the data, can you please be more
> explicit as to the need?]

This is so that we are certain that when an application is
running, it will display nicely.  If the hardware has only
15bpp, the client would want to know.  If the client wants
to paint a 15bpp image, the server would want to know (In
case of translation).  Another small difference could be
the bit packing order (This is why I named GrArea_R5G6B5
and not GrArea_565).  This will help avoid any
misunderstandings of what the function does, and allow for
slightly different formats to be supported in the future
using the same naming convention.

For the transparent 16bpp Area function (where the client
produces pixels with the same format as the hardware),
we may use a generic function like GrArea_16bpp in the server
hooked up to the GrArea_R5G6B5 opcode since memcpy to the
frame buffer is not dependent on the pixel format.

> [Also, we don't want to query
> the opcodes, since, with the new client/server rewrite,
> almost all opcodes return void, so that they can be buffered
> and not have to wait for a return value]

No, the SCREENINFO should be used to decide which rendering
algorithm to be used in the client if several are available,
or decide that the client can't run if it uses one or more
unsupported pixel formats.

Once the action starts, you will want to differentiate between
15bpp and 16bpp requests since the packing is physically
different and the server needs to know this if translation is
needed (assuming it is supported, as reported by SCREENINFO).

> : Well, not if the server uses the same definition for PIXELVAL,
> : but this is a fact I don't trust, and having a Nano-X server
> : not being able to detect an unsupported condition is bad IMHO.
> : To be able to detect unsupported use, extra opcodes are
> : needed, and a stub function to return, or report, an error for
> : the unsupported function.
> 
> Like I mentioned above, we definitely want to use SCREENINFO
> for this, and not change the type of server procedure to return
> something other than void.

Yes! I don't want to change the protocol, but we don't want
to segfault either if a protocol opcode is not supported
(A program may incorrectly interpret the SCREENINFO and try
to use some unsupported functionality).  We can just silently
ignore it or log it in the server log.

Regards,
- Morten

: IMO the runtime version is only needed when there is a chance that the
: application may be run on a server that doesn't support the mode used
: when compiling the application.  This is something we may want to avoid,
: but when it can't be avoided, the server can translate if available and
: enabled.  The server has the benefit of being able to allocate translation
: tables only once, and not once for each client which would be the case for
: client side conversion.
:
: I much more favor the idea of having the server translate if the need
: arises, and guarantee that a certain 24 bit true color format will
: always be supported in the server (possibly by translation).  This
: will be both portable and provide the best quality (at a memory and
: performance penalty).
: 
: This is a special case of what I propose; Guarantee that the server
: will translate the 24bpp pixel format correctly onto whatever screen
: used, everywhere.
: 
: For a lot of programs this is fine, for some that needs every available
: cycle or memory it is a little bit more problematic, so we provide a
: way to create 8 or 16 bit pixels *fast* and ship them over.
: 
: For autonomous systems, this is just great.  If we later find that
: we need to support 16 bit only apps on 24 bit color etc., we have the
: option of letting the server translate.
:
: Giving the users a choice of using the format always available if
: porting is a concern, and hit the iron "directly" if speed/space
: is important, would be very nice.  If the user wants to support both
: formats (or even more) in the same application, he is free to do so.
:
Whew!

Well, after looking at all this, and thinking about it, I've got a solution
that
I think will meet your needs, and keep the project small, tight,
and fast.  And extensible.

First, we stay with a single, modified GrArea() call.  The modification
is to pass a single PF_xxx type with GrArea that tells the server
what type of data is being passed.  This keeps alot of duplicate
code out of the server, and keeps the opcodes down, which
aren't really needed.

Second, we use your good idea of stating that the server will
always translate certain pixel formats.  Now, here's the tricky part.
I suggested that RGB COLORVALs be always translatable.  You
want TRUECOLOR888.  Well, the only difference between these
two is actually a 24bpp vs 32bpp packing.  So it's not a big deal,
and the server will always call GdFindColor on the data (which
isn't fast, BTW).  We add a PF_COLORVAL and rename
PF_TRUECOLOR24 to PF_TRUECOLOR888 to make this work.

Now, here's the other part.  The PF_xxx values can be rewritten
as mask values, and the server can return via SCREENINFO
the entire set of masks that it will allow, or translate.  This
allows a client application to specify a hardware PIXELVAL
value directly in a GrArea() call.  Here's the important part:
if you want speed, there's only *one* way we're going to get it.
That is, the client builds an image packet, it gets queued,
finally sent over, and then the server *must* call the screen
driver bitblit routine, it CANNOT muck with the data.  Note,
currently, that's NOT the way GrArea works.  GrArea unpacks
the data and calls DrawPixel.  In my earlier work with
bitblit, added recently, I found an order of magnitude
difference between, believe it or not, a hand-coded
for() loop and an inline memset() libc call.   So, for
speed, the client's got to build the hardware pixel
values, period.  Of course we don't demand this,
since PF_COLORVAL and PF_TRUECOLOR888 will
always be translated.

Anybody can add PF_xxx translations to the GrArea server
entry point, and convert data, although this
will never provide speed.  This allows say, 16bpp
clients to talk with 24bpp hardware.

I supply some macros to convert between
PIXELVALs and COLORVALs, and the client
or server implementor is free to use them as
desired.


I think this design covers all your above points.

Regards,

Greg

Hello Greg,

Your latest changes seems very nice!  I/we have not been able to test it
yet, but will do.

> First, we stay with a single, modified GrArea() call. The modification
> is to pass a single PF_xxx type with GrArea that tells the server
> what type of data is being passed.  This keeps alot of duplicate
> code out of the server, and keeps the opcodes down, which
> aren't really needed.

Yes.  The GrArea will not spend most of its time in the switch
logic anyway I guess...

> Second, we use your good idea of stating that the server will
> always translate certain pixel formats.  Now, here's the tricky part.
> I suggested that RGB COLORVALs be always translatable.  You
> want TRUECOLOR888.

Oh, well. I'm not sure if I really wanted 3-byte packing of the
24-bit case.... I must have called it by the wrong name by accident.
It will reduce the size of the request, at the expense of
simplicity when building the image at the client.  Don't settle
for 3-byte packing if you think 4-byte alignment is better;
I will probably not use the portable case at all anyway.

> Well, the only difference between these
> two is actually a 24bpp vs 32bpp packing.  So it's not a big deal,
> and the server will always call GdFindColor on the data (which
> isn't fast, BTW).

If we choose wrong, we may not be able to utilize accels at a
later stage when optimizing.  I can ask at the GGI mailing list
what direction hardware is heading WRT 3-byte/4-byte 24bpp layouts,
I'm fairly sure they can tell me more than I want to know
about this issue:-)

> We add a PF_COLORVAL and rename
> PF_TRUECOLOR24 to PF_TRUECOLOR888 to make this work.

As said, don't change anything in this respect fro my sake only;
I would hate to realize I have prevented "the right thing" from
happening!

> Now, here's the other part.  The PF_xxx values can be rewritten
> as mask values, and the server can return via SCREENINFO
> the entire set of masks that it will allow, or translate.  This
> allows a client application to specify a hardware PIXELVAL
> value directly in a GrArea() call.  Here's the important part:
> if you want speed, there's only *one* way we're going to get it.
> That is, the client builds an image packet, it gets queued,
> finally sent over, and then the server *must* call the screen
> driver bitblit routine, it CANNOT muck with the data.  Note,
> currently, that's NOT the way GrArea works.  GrArea unpacks
> the data and calls DrawPixel.  In my earlier work with
> bitblit, added recently, I found an order of magnitude
> difference between, believe it or not, a hand-coded
> for() loop and an inline memset() libc call.

I hope the memset was the faster one, no? :-)

> So, for
> speed, the client's got to build the hardware pixel
> values, period.  Of course we don't demand this,
> since PF_COLORVAL and PF_TRUECOLOR888 will
> always be translated.

Yes!

> Anybody can add PF_xxx translations to the GrArea server
> entry point, and convert data, although this
> will never provide speed.  This allows say, 16bpp
> clients to talk with 24bpp hardware.

We should also take into consideration the question on
differing pixel packing formats when naming the
macros, but I guess a non-standard 565 encoding could
just get, say, '_nonstandard' attached to the macro name:-)

The other commonly used format IIRC, 555 can be named
consistently anyway.

> I supply some macros to convert between
> PIXELVALs and COLORVALs, and the client
> or server implementor is free to use them as
> desired.
> 
> I think this design covers all your above points.

Yes, I think so too.  I'll look into it as soon as I get
the chance.  Thanx!

- Morten

: Oh, well. I'm not sure if I really wanted 3-byte packing of the
: 24-bit case.... I must have called it by the wrong name by accident.
: It will reduce the size of the request, at the expense of
: simplicity when building the image at the client.  Don't settle
: for 3-byte packing if you think 4-byte alignment is better;
: I will probably not use the portable case at all anyway.

I'm going to add a PF_TRUECOLOR0888 that will specify
4-byte packing for 8/8/8 pixels.  Is this what you plan to use?



: > We add a PF_COLORVAL and rename
: > PF_TRUECOLOR24 to PF_TRUECOLOR888 to make this work.
: 
: As said, don't change anything in this respect fro my sake only;
: I would hate to realize I have prevented "the right thing" from
: happening!

What's the right thing that's not happening?

: 
: > So, for
: > speed, the client's got to build the hardware pixel
: > values, period.  Of course we don't demand this,
: > since PF_COLORVAL and PF_TRUECOLOR888 will
: > always be translated.

In 0.87pre3, PF_COLORVAL is translated, nothing else is,
but instead is just unpacked and passed to the screen driver.


Regards,

Greg