Software Configuration and Description: Magic Leap

The Magic Leap can be utilized with game engines, such as Unity and Unreal, and also directly through the SDK using C or C++. Magic Leap provides a lot of information on their development web pages. Some additionl information is provided here as well.

In order to run custom software on the Magic Leap 1, that software needs to be installed on the device. Alternatively, you can use Magic Leap's Zero Iteration, for example, within Unity. To communicate with the Magic Leap 1, a USB connection can be used, for example, by using a USB-A to USB-C cable. The Magic Leap 1 will prompt to ask for permission before any communication can take place. These permissions may have been already granted previously.

You can also communicate with the Magic Leap 1 over a WiFi connection. For this to work, you will have to know the device's IP address and the service needs to be activated on the Magic Leap 1. In order to set up the device, Magic Leap's software "The Lab" provides a command line interface that can also be used for a variety of other communication and configuration with the device. In order to access the command line interface, start The Lab software and activate the sidebar on the left. The command line interface is accessible from the icon at the very bottom of the sidebar. To activate WiFi communication, you can use the following two commands. The full list of commands available for the Magic Leap Device Bridge (mldb) interface is available here.

• mldb wifi status: this will print out the IP address (among other information)
• mldb tcpip -p 5555: this will start the TCP server on the Magic Leap 1 at port 5555

More detailed information is available from Magic Leap directly. It is important to note, that the device needs to be connect via USB in order for those commands to be issued. The devices in the teaching lab should already be configured properly for you. You can also check the IP address on the Magic Leap 1 directly by selecting the WiFi icon and the information icon for the active WiFi network.

The computers in the teaching lab are dual boot, i.e. you can switch between Linux and Windows. The Windows computers have the necessary software installed to develop software for the Magic Leap 1. You can connect to the Magic Leap 1 over the Wireless network or by using the USB-C connection. The USB-C connection is required for Zero Integration, which allows you to run your software directly on the Magic Leap 1 without the need to install it on the device. Instead, you can simply start it within the Unity environment which is great for software development. Alternatively, a preconfigured virtual machine running Windows 7 or Windows 10 are ready for you to use on the Linux computers in the teaching lab that can be used to develop software for the Magic Leap 1. You can start those by issuing the command "windows" or "windows -10", repsectively. The computer are also can be booted into Windows directly using the grub boot manager. Most of the necessary software is already installed for those computers and virtual machines. Any changes you make to those configurations will not be permanent, i.e. will be lost on reboot. If you need a permanent version of the virtual machine, you can copy the image to a local device, such as a USB drive, and then start the virtual machine with "windows -i <path-to-image>". If you decide to use Linux, you can only connect to the Magic Leap 1 over the network.

Developing Software for the Magic Leap 1

In order to deploy software on the Magic Leap 1, all software needs to be certified. The required certificates are available from Magic Leap. A generic certificate is already available in the Documents folder. To get started more quickly, Magic Leap made a basic template and a toolkit available for Unity to assist with the software development. This toolkit can be found here. Further, this package provides additional useful items. A description for some of the features available within this package can be found in this PDF. There is also a troubleshooter to make tracking down issues a little easier. When you start with the basic template, Unity will require you to switch the build platform from the default to Lumin so it knows that you are trying to build an application for the Magic Leap 1.

Setting up Zero Integration

Zero Integration is a feature by Magic Leap that makes software development a little easier. Instead of compiling the binary within Unity, installing it on the Magic Leap 1, and then starting the application on the device, Zero Integration is used to directly run the application within Unity on the device. All log entries you print out (via Debug.Log, for example) will show up in the Unity console (otherwise you will have to use the logging feature available within the The Lab's Device Bridge to see throse print outs).

In order to use Zero Integration, a few things have be set up first. The following list includes every step necessary:

• Switch the platform to Lumin, you may have to set the path to the Lumin/Magic Leap SDK in External Tools.
• Import support libraries in Unity: Magic Leap->ML Remote->Import Support Libaries
• Launch Zero Integration from within Unity: Magic Leap->ML Remote->Launch Zero Integration
• Enable to connected Magic Leap 1 device
• Hit the play button in Unity

Installing an application on the Magic Leap 1

There are two ways to install an application compiled with Unity (or through other mechanisms) on the Magic Leap 1. The more complicated way is to use the Magic Leap Device Bridge (mldb) via the command line interface. The other option is the use Magic Leap's The Lab. The Device Bridge lets you see the status of the Magic Leap device and all installed application. It also enables you to install and uninstall applications by using the Install App and Uninstall buttons, respectively.

Using the controller as input device

Dealing with the standard input device of the Magic Leap 1 is relatively simple. There are some basic scripts and prefabs already available in the Magic Leap Toolkit, such as ControlPointer and ControlInput. Depending on the version of the toolkit you are using, some of those tools fail to initialize the control pointer instance properly requiring you to add a line like the following line at the beginning of the start method:

	controlInput = getComponent<ControlInput>();
    

As the samples from the toolkit make you rely more on the features available in Unity and the Unity Editor, it may be easier to just code the handling of the input device directly in your own script. I found this tutorial very helpful to figure out how to write your own scripts. Additional sections are available that describe how to get access to all the buttons and trigger on the input device, as well as how to obtain 6 degrees-of-freedom input and how to use ray casting to select objects from within the AR application using the input device. The following example shows a simple script I wrote to use the controller to drag an object around by activating the trigger:

    
	using System.Collections;
	using System.Collections.Generic;
	using UnityEngine;
	using UnityEngine.UI;
	using UnityEngine.Events;
	using UnityEngine.XR.MagicLeap;

	public class MyControlHandler : MonoBehaviour
	{
	    #region Private Variables
	    private GameObject sphere;
	    private Quaternion originalOrientation;
	    private Vector3 originalPosition;
	    private Vector3 StartPosition;
	    private Quaternion StartQuaternion;
	    private Vector3 rotation = new Vector3(0, 0, 0);
	    private const float rotationSpeed = 30.0f;
	    private MLInput.Controller controller;
	    private bool Button = false;
	    private float lastHomeTime;
	    private readonly float homeDoubleDuration = .5f;
	    private float triggerLastTime;
	    private readonly float triggerDoubleDuration = .5f;
	    private bool Trigger = false;
	    #endregion

	    // Start is called before the first frame update
	    void Start()
	    {
	        Debug.Log("Input handler started...");
	        sphere = GameObject.Find("Sphere");
	        originalOrientation = sphere.transform.rotation;

	        MLInput.Start();
	        MLInput.OnControllerButtonDown += OnButtonDown;
	        MLInput.OnControllerButtonUp += OnButtonUp;
	        MLInput.OnTriggerDown += OnTriggerDown;
	        MLInput.OnTriggerUp += OnTriggerUp;
	        controller = MLInput.GetController(MLInput.Hand.Left);
	    }

	    // Cleans up event handlers and stops receiving input
          void OnDestroy()
          {
              MLInput.OnControllerButtonDown -= OnButtonDown;
              MLInput.OnControllerButtonUp -= OnButtonUp;
              MLInput.OnTriggerDown -= OnTriggerDown;
              MLInput.OnTriggerUp -= OnTriggerUp;
              MLInput.Stop();
          }

          // Update is called once per frame
          void Update()
          {
              if (Trigger)
              {
                  sphere.transform.position = originalPosition + controller.Position - StartPosition;
                  Quaternion diff = Quaternion.identity * Quaternion.Inverse(StartQuaternion) * Quaternion.Inverse(Quaternion.identity * Quaternion.Inverse(controller.Orientation));
                  sphere.transform.rotation = diff * originalOrientation;
              }
          }

          void OnButtonDown(byte controllerId, MLInput.Controller.Button button)
          {
              if (button == MLInput.Controller.Button.Bumper)
              {
              }
          }

          void OnButtonUp(byte controllerId, MLInput.Controller.Button button)
          {
              switch (button)
              {
                  case MLInput.Controller.Button.Bumper:
                      break;
                  case MLInput.Controller.Button.HomeTap:
                      //double?
                      Debug.Log("Home tap" + (Time.realtimeSinceStartup - lastHomeTime) + " " + lastHomeTime);
                      if (Time.realtimeSinceStartup - lastHomeTime < homeDoubleDuration)
                      {
                          ExitApp();
                      }

                      lastHomeTime = Time.realtimeSinceStartup;
                      break;
              }
          }

          private void OnTriggerDown(byte controlId, float triggerValue)
          {
              Debug.Log("trigger pulled" + triggerValue);
              if (triggerValue > 0.5)
              {
                  if (!Trigger)
                  {
                      StartPosition = controller.Position;
                      StartQuaternion = controller.Orientation;
                      originalOrientation = sphere.transform.rotation;
                      originalPosition = sphere.transform.position;
                  }
                  Trigger = true;

                  //double?
                  if (Time.realtimeSinceStartup - triggerLastTime < triggerDoubleDuration)
                  {
                  }

                  triggerLastTime = Time.realtimeSinceStartup;
              }
          }

          private void OnTriggerUp(byte controlId, float triggerValue)
          {
              Trigger = false;
          }
          public void ExitApp()
          {
      #if UNITY_EDITOR
              Debug.Log("Exit to Home: Unity EditorQuit");
              UnityEditor.EditorApplication.isPlaying =false;
      #else
              Application.Quit();
      #endif
          }
      }      
    
    

The Magic Leap 1 also supports hand recognition and voice commands for even more sophisticated control of your application.

  

  ©2005-2022 Thomas Wischgoll