Cheat Sheet:
Unicode-enabling Microsoft C/C++ Source Code

Initial Steps for Unicode-enabling Microsoft C/C++ Source

• Define _UNICODE, undefine _MBCS if defined.
• Convert literal strings to use L or _T
• Convert string functions to use Wide or TCHAR versions.
• Clarify string lengths in API as byte or character counts. For character-based display or printing (as opposed to GUI which is pixel-based) use column counts, not byte or character.
• Replace character pointer arithmetic with GetNext style, as characters may consist of more than one Unicode code unit.
• Watch buffer size and buffer overflows- changing encodings may require either larger buffers or limiting string lengths. If character size changes from 1 byte to as many as 4 bytes, and string length was formerly 20 characters and 20 bytes, either expand the string buffer(s) from 20 to 80 bytes or limit the string to 5 characters (and therefore 20 bytes). Note maximum buffer expansion may be constrained (for example to 65 KB). Reducing string length to a fixed number of characters may break existing applications. Limiting strings to a fixed byte length is dangerous. For example, allowing any string that fits into 20 bytes. Simple operations such as uppercasing a string may cause it to grow and exceed the byte length.
• Replace functions that accept or return arguments of a single character, with functions that use strings instead. (International) Operations on a single character may result in more than one code point being returned. For example, upper('ß') returns "SS".
• Use wmain instead of main. The environment variable is then _wenviron instead of _environ.
  wmain( int argc, wchar_t *argv[ ], wchar_t *envp[ ] ).
• MFC Unicode applications use wWinMain as the entry point.
  In the Output page of the Linker folder in the project's Property Pages dialog box, set the Entry Point symbol to wWinMainCRTStartup.
• Consider fonts. Identify the fonts that will render each language or script used.

File I/O, Database, Transfer Protocol Considerations

• Consider whether to read/write UTF-8 or UTF-16 in files, databases, and for data exchange.
• Consider Endian-ness in UTF-16 files.
  Read/Write Big-Endian on networks. Use Big-Endian if you don't produce a BOM.
  Endian-ness of files will depend on the file format and/or the architecture of the source or target machine.
  When reading files encoded in UTF-16 or UTF-32, be prepared to swap-bytes to convert endian-ness.
  Also consider streams and transfer protocols and the encoding used in each.
• Label files or protocols for data exchange with the correct character encoding. E.g. set HTTP, HTML, XML to UTF-8 or UTF-16.
• Consider Unicode BOM (Byte Order Marker) and whether it should be written with data. Remove it when reading data.
• Consider encoding conversion of legacy data and files, import and export, transfer protocols. (MultiByteToWideChar, WideCharToMultiByte, mbtowc, wctomb, wctombs, mbstowcs )
• Consider writing to the Clipboard-
  use CF_TEXT format and write native character encoding (ANSI) text, and
  use CF_UNICODETEXT format and write Unicode text.
• Database applications should consider Data Type (NCHAR, NVARCHAR) and Schema Changes, Triggers, Stored Procedures, and Queries. Data Storage growth, Indexes and Performance.
  Note that the Unicode schema changes will have different impacts and concerns on different vendors' databases. If database portability is a requirement, the features and behaviors of each database need to be taken into account.
  (I know this item is seriously understated. To be expanded sometime in the future.)

Stream I/O

Streams are difficult in Microsoft C++. You may run into 3 types of problems:

1. Unicode filenames are not supported. The workaround is to use FILE * _wfopen and if needed, use the FILE handle in subsequent stream I/O.
   std::ifstream stm(_wfopen(pFilename, L"r"));
2. Stream I/O will convert Unicode data from/to native (ANSI) code page on read/write, not UTF-8 or UTF-16. However the stream class can be modified to read/write UTF-8. You can implement a facet to convert between Unicode and UTF-8.
   codecvt <wchar_t, char_traits <wchar_t> >
3. To read/write UTF-16 with stream I/O, use binary opens and binary I/O. To set binary I/O:
   _setmode( _fileno( stdin ), _O_BINARY );
   
   Also see the Microsoft run-time library reference: "Unicode Stream I/O in Text and Binary Modes".

Note: There aren't TCHAR equivalents for cout/wcout, cin/wcin, etc. You may want to make your own preprocessor definition for "tout", if you are compiling code both ways.

Internationalization, Advanced Unicode, Platform and Other Considerations

• Consider using locale-based routines and further internationalization.
• For Windows 95, 98 and ME, consider using the Microsoft MSLU (Microsoft Layer for Unicode)
• Consider string compares and sorting, Unicode Collation Algorithm
• Consider Unicode Normalization
• Consider Character Folding
• Reconsider doing this on your own. Bring in an experienced Unicode consultant, and deploy your existing resources on the tasks they do best. (Hey, an I18nGuy's gotta earn a living...)

Constant and Global Variables

Data Types

Platform SDK String Functions

There are many Windows API that compile into ANSI or Wide forms, depending on whether the symbol UNICODE is defined. Modules that operate on both ANSI and Wide characters, need to be aware of this. Otherwise, using the Character Data Type-independent name requires no changes, just compile with the symbol UNICODE defined.

The following list is by no means all of the Character Data Type-dependent API, just some character and string related ones. Look in WinNLS.h for some code page and locale related API.

TCHAR String Functions

Functions sorted by ANSI name, for ease of converting to Unicode.