Traditionally, game developers have been constrained in their networking choices. TCP and HTTP are typically far too slow for any seriously real-time game, leaving UDP and all of the… interesting problems that go along with it. Or WebSockets, for the truly masochistic.
GRPC, or Google’s take on Remote Procedure Calls, are a fast and efficient use of HTTP/2. I had a dream of being able to implement both my client and server in the same environment (Unity / C#), using GRPC for real-time push/pull connections, while still being able to fall back on HTTP/1.1 and serve traditional RESTful and/or GraphQL web traffic from the backend. I’ll cover many other parts of this elsewhere, but in this post I’ll look at getting GRPC to cross-compile such that my Unity build would still be able to target all viable platforms.
Aside: it’s been a long time since I worked with compilers, Makefiles, and the like. Please excuse my inevitable misuse of some terms.
The Problem
GRPC already works on pretty much every imaginable platform. The problem comes when you build a Unity project, which likely depends on the C# implementation. The result is an executable which depends on the C# library rather than the appropriate library for the platform. This is a problem for iOS because it requires a static library (.a), whereas the compiled Unity binary yields a dynamic library (.dylib).
The Solution
I was ecstatic to find that the creator of MagicOnion already documented the solution on his blog (now offline). Unfortunately it’s in Mandarin Chinese (and more than a few GRPC versions behind), so I’ll document my process for implementing a similar solution here.
In a nutshell, the idea is to compile the open-source GRPC project as a static library, in this case for arm64. Then, you can simply copy-and-paste the requisite C# files into the Unity project and modify them to invoke the static library. My case was also a bit unique in that I am creating a game library (VisualStudio C# DLL) to be used in Unity for my different game targets. This meant that I didn’t have access to preprocessor macros and build flags which might otherwise be used to flip certain configurations.
Building the Static LibrarIES
If you haven’t already, clone and set up the GRPC repository:
git clone git@github.com:grpc/grpc.git cd grpc git submodule update --init
You may also need to follow other steps enumerated in their README, depending on platform. I’d also recommend checking out the release version of GRPC equal to the current C# production version available in the Nuget Grpc.Core package (1.8.3 as of this writing), instead of the default development branch.
libgrpc.a
Once you have a directory which can build (i.e., make succeeds), the first step is to modify the Makefile in the root of the GRPC repository. Take note of the path to your iPhone SDK; you’ll need to add an IOSFLAGS variable to the Makefile. Look for where CC_opt and CXX_opt are defined and change the lines to look something like this:
IOSFLAGS = -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk -fembed-bitcode CC_opt = $(DEFAULT_CC) $(IOSFLAGS) CXX_opt = $(DEFAULT_CXX) $(IOSFLAGS)
Next, remove the -Werror flag from where it is appended to the CPPFLAGS variable. The line should now look something like:
CPPFLAGS += -g -Wall -Wextra -Wno-long-long -Wno-unused-parameter -DOSATOMIC_USE_INLINED=1 -Wno-deprecated-declarations
Now you should be able to run the make command again. There may be some errors, but you should end up with ./libs/opt/libgrpc.a which you can copy into the Plugins/iOS directory of your Unity project. Unity will detect this as a static library applicable to iOS.
grpc_csharp_ext.a (iOS)
iOS needs a statically linked .a file, built with:
gcc -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.2.sdk -I. -I./include -c -o grpc_csharp_ext.o src/csharp/ext/grpc_csharp_ext.c ar -rsc grpc_csharp_ext.a grpc_csharp_ext.o
Which yields grpc_csharp_ext.a, which should be copied in to Unity’s Plugins/iOS directory.
libgrpc_charp_ext.so (Android)
Android needs a statically linked .so file; see my discussion here with the GRPC team. You’ll need to make a small tweak to CMakeLists.txt, to change the _gRPC_ALLTARGETS_LIBRARIES in Unix to not include rt or pthread. In my case, it now looks like:
if(_gRPC_PLATFORM_MAC)
set(_gRPC_ALLTARGETS_LIBRARIES ${CMAKE_DL_LIBS} m pthread)
elseif(UNIX)
set(_gRPC_ALLTARGETS_LIBRARIES ${CMAKE_DL_LIBS} m)
endif()if(_gRPC_PLATFORM_MAC)
set(_gRPC_ALLTARGETS_LIBRARIES ${CMAKE_DL_LIBS} m pthread)
elseif(UNIX)
set(_gRPC_ALLTARGETS_LIBRARIES ${CMAKE_DL_LIBS} m)
endif()Next, I had to modify log_android.cc to comment out the use of __android_log_write due to linker errors at runtime. Instead, below, I’ll demonstrate a modification to NativeLogRedirector.cs that pipes into my game’s log/error pipeline. From there, assuming that the Android NDK is installed at $ANDROID_NDK, run:
cmake \ -DCMAKE_SYSTEM_NAME=Android \ -DCMAKE_SYSTEM_VERSION=15 \ -DCMAKE_ANDROID_ARCH_ABI=armeabi-v7a \ -DCMAKE_ANDROID_NDK=$ANDROID_NDK \ -DCMAKE_ANDROID_STL_TYPE=c++_static \ -DRUN_HAVE_POSIX_REGEX=0 \ -DRUN_HAVE_STD_REGEX=0 \ -DRUN_HAVE_STEADY_CLOCK=0 \ -DCMAKE_BUILD_TYPE=Release make grpc_csharp_ext EMBED_OPENSSL=true
This generates the libgrpc_csharp_ext.so, which should be copied into Unity’s Plugins/Android directory.
Now that you’ve got the static libraries, it’s time to modify Grpc.Core.
Modifying the C# Code
You can effectively copy all of the .cs files from the C# Grpc.Core directory, including those in all sub-directories except Properties. I added these to my VisualStudio project while removing the Grpc.Core Nuget dependency, and everything complied as-expected.
Loading Static Libraries
There’s a lot of copy-and-paste here, since the goal is to add DllImport commands for the static library. I added a new class, InternalLib, that looks something like this…
using System;
using System.Runtime.InteropServices;
namespace Grpc.Core.Internal {
internal static class InternalLib {
private const string pluginName = "__Internal";
[DllImport(pluginName)]
internal static extern void grpcsharp_init();
[DllImport(pluginName)]
internal static extern void grpcsharp_shutdown();
// ... and so onEach of these methods are copied directly from NativeMethods.cs, with the help of a little Regex find-and-replace foo. This gives a way to access the static library, but now we need to modify the existing code to use it. This is where my approach differs a bit from that of MagicOnion. Instead of using compile flags to switch the build type, I attempted to build a single DLL from my .NET project which could gracefully handle either eventuality. To this end, I modified the return for the Load() method of NativeExtensions.cs quite simply:
return paths.Any(File.Exists) ? new UnmanagedLibrary(paths) : null;
Using Linq, this simply returns a null value when the dynamic library cannot be found, instead of throwing an error when the UnmanagedLibrary is instantiated. Now, the constructor to the the NativeMethods class can/will receive a null value for the library. In this case, I attempt to fall back on the static library. Perhaps the “cleanest” way to do this would be to modify the GetMethodDelegate function (which looks up the method in the UnmanagedLibrary) to also support loading from the InternalLib class. However, the reflection involved in transforming to the correct return type was delicate, and I preferred the compile-time safety of simply using a hardcoded ternary for the time being…
public NativeMethods(UnmanagedLibrary library) {
this.grpcsharp_init = library == null ? StaticLib.grpcsharp_init : GetMethodDelegate<Delegates.grpcsharp_init_delegate>(library);
this.grpcsharp_shutdown = library == null ? StaticLib.grpcsharp_shutdown : GetMethodDelegate<Delegates.grpcsharp_shutdown_delegate>(library);
// ...and so on...Other Tweaks
The MagicOnion blog post I mentioned also suggests small tweaks to two other files.
The first change is to make the logging in NativeLogRedirector.cs to use the UnityEngine’s logger instead of Console. In my case, I actually have my own logging tooling built-in to my .NET library to handle different platforms. Therefore my tweak will not be relevant to others.
The second change to effectively disable certificate pinning in DefaultSslRootsOverride.cs. I have not yet determined if this is absolutely necessary, but I did include the change in my build for the time being.
Building For Devices
With the two static libraries copied into the Plugins/iOS directory, the code should now compile and work on iOS. The only caveat is that you need to link the Xcode project against libz.dylib for gzip compression.
Linux, Mac, and Windows should “just work…” with the caveat that you may need to manually copy the grpc_csharp_ext.dylib file into the correct location in the output directory. You’ll be able to determine where GRPC is looking for this dylib based upon errors in the player logs. I believe Web should also work with the correct static libraries, but have not finished testing yet.
Overall, this was a great proof-of-concept that GRPC would indeed work across all our target platforms despite being cross-compiled in Unity.
