Wednesday, February 27, 2013

SQLite3 in Ruby 1.9: be sure your Ruby data is UTF-8

SQLite3 is not finding your records? SQLite3 is inserting binary data instead of plain-text strings? If you're using Ruby 1.9 or later, you must be sure your Ruby data is encoded as UTF-8 before submit it to SQLite3.

Text data in SQLite is UTF-8. Before Ruby 1.9, Strings had no encoding; so that was not a issue; all data get same handling. From Ruby 1.9, Strings must be encoded as UTF-8 to be compatible with SQLite data; otherwise your text will be considered binary content to SQLite, and binary content won't match plain-text content, even when they are the very same bytes.

Maybe the source of your text data encodes them as other than UTF-8. To make sure your data is UTF-8, try using String#encode or String#force_encoding before submit it to SQLite 3. Cheers!