Custom-shaped Win32 Window with chroma keying

Creating custom shaped and semi-transparent windows using modern programming techniques (such as managed code or visual-drag-and-drop IDE’s) has become a trivial task. But sometimes due to your project’s requirements, performance constraints or simply because you love making your GUI’s from scratch, those tools are not an option.

In this article I’ll show how to code a custom-shaped window with chroma keying using our beloved Win32 API. Some familiarity with Win32 GUI coding is suggested (although you can always just copy-paste the code). This feature is available from Windows 2000 Professional onward.

The final result
The final result

Chroma Keying, as you may already know, is a technique that consists of replacing a particular color in an image for another thing, or , like in this case, making it transparent.

That’s why we’ll start off by creating the image we’ll be using as our window. You can pick the image editor of your choice, just make sure that those areas you want to be transparent are set to a precise color that is not repeated elsewhere. I’m using green RGB(0,255,0) but you can select whatever you want. Save your file as 24-bit Windows BMP file.



Now, let’s get into the code. The first part is pretty much the standard procedure to initialize a window except that we neeed to create a brush from our bitmap and assign it as the background of the window in the class defintion:

 //create a brush from a bitmap to use it as window background
 HBRUSH brush = CreatePatternBrush((HBITMAP)LoadImage(NULL,windowBitmapPath,IMAGE_BITMAP, 0, 0, LR_LOADFROMFILE));
 //Define and register the window class
 WNDCLASSEX windowClass;

windowClass.hbrBackground = brush;

Next we create our window handle using CreateWindowEx() since we’ll be setting the WS_EX_LAYERED style. To avoid the window frame to be visible we’ll use WS_POPUP style as well.

//We use WS_EX_LAYERED and WS_POPUP to obtain the desired effect
HWND windowHandle = CreateWindowEx(WS_EX_LAYERED,

The next step is where the magic actually happens. First we create a COLORREF variable to specify the transparent color (using the RGB( ) macro).

By setting LWA_COLORKEY, SetLayeredWindow() tells the window to use the COLOREF RGB color as the transparent one with the alpha value specified on the third parameter (0 in this case,  meaning fully transparent).

COLORREF transparentColor = RGB(0,255,0);
 SetLayeredWindowAttributes(windowHandle, transparentColor,0,LWA_COLORKEY);

Two more things and we’re ready to go.

First, since we’re using WS_POPUP as our window style there is no caption to drag it from. It is useful to add the capability of dragging by the client area. That can be easily achieved by handling the WM_NCHITTEST inside the window’s WndProc( ).

 if (DefWindowProc(hWnd, msg, wParam, lParam) == HTCLIENT)
 return HTCAPTION;

What this code does is telling the window to handle HTCAPTION instead of HTCLIENT when the client area is clicked.

Lastly, it is a good idea to add custom buttons to Close/Minimize the window.

//ID for the close button
#define BUTTON_ID 0x01

( ... )

 HWND closeButton = CreateWindow(L"button",NULL,WS_CHILD|WS_VISIBLE|BS_DEFPUSHBUTTON|BS_BITMAP,481-18,0,18,16,hWnd,(HMENU)BUTTON_ID,GetModuleHandle(NULL),NULL);

( ... )

 switch (wParam)
 case BUTTON_ID:


( ... )

And that’s it! You can download the visual studio solution for this example (including the sample images) from here  .

Source code is distributed under GPL v3.o license.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s