Cython 3.1.0a0 Documentation – Unicode and passing strings

in

That’s where Cython comes in it lets us write fast, compiled code with Python syntax. But what about strings? How do we handle them without sacrificing performance or sanity?

Well, let me tell you all about Unicode and passing strings in Cython 3.1.0a0! (I know, I’m getting excited too.)

First there are four types of Python strings that Cython supports: bytes, str, unicode, and basestring. The last one is a catch-all for both the str and unicode types in Python 2, but we won’t be using it much here since we’re focusing on Cython 3.1.0a0.

In this version of Cython, you cannot assign a Unicode string to a variable or argument that is typed as ‘str’. This can lead to compile time errors or TypeErrors at runtime if you’re not careful. If your code needs to be compatible with Python 2 (which allows mixing byte strings and unicode strings), type variables and arguments as either bytes or basestring instead of str.

But what about passing C strings? Cython supports two ways: the first is by using the ‘c_string_type’ directive, which automatically converts between Python and C strings in simple cases. The second way involves decoding bytes to text on reception and encoding text to bytes on output. Let me show you an example of each method!

First, let’s say we have a function that returns a C string:

# Import the necessary module
from c_func import c_call_returning_a_c_string

# Define the main function
def main():
    # Call the c_call_returning_a_c_string function and assign the returned C string to the variable cstr
    cstr = c_call_returning_a_c_string()
    
    # Do something with the C string...
    
    # Decode the C string to a Python Unicode string using the 'UTF-8' encoding and assign it to the variable ustr
    ustr = cstr.decode('UTF-8')

Notice that we’re using ‘bytes.decode()’ to convert the C string to a Python Unicode string, but only after checking for null bytes (which would cause an error). If you know your function won’t return any null bytes, you can skip this step and just pass the raw C string directly to Cython code that expects it as a byte string.

Now let’s say we have another function that takes a Python Unicode string:

# Import the c_func module which contains the c_call_taking_a_unicode_string function
import c_func 

# Define the main function
def main():
    # Create a Python Unicode string
    ustr = 'hello, world!'
    
    # Encode the string to a C byte string using UTF-8 encoding
    cbytes = ustr.encode('UTF-8')
    
    # Call the c_call_taking_a_unicode_string function from the c_func module, passing in the encoded bytes
    c_func.c_call_taking_a_unicode_string(cbytes)

# Call the main function to execute the code
main()

# Note: This script assumes that the c_func module has been properly imported and the c_call_taking_a_unicode_string function is defined within it.

Again, we’re using ‘bytes.encode()’ to convert a Python Unicode string to a C byte string before passing it to Cython code that expects it as a byte string. This ensures that our function receives the correct data in the expected format.

And there you have it handling strings in Cython 3.1.0a0! It’s not always easy, but with these tips and tricks, you can write fast, compiled code without sacrificing readability or maintainability.

SICORPS