QueTwo's Blog

thouoghts on telecommunications, programming, education and technology

Using AIR Native Extensions for Desktop and Mobile

During Wednesday’s meeting of the Michigan ActionScript User Group, we covered what AIR Native Extensions are, where to find the best ones, and how to actually use them.  Includes demos from both Desktop AIR and Mobile AIR projects.

The two locations to find some of the more popular ANEs are :

Creating a Windows AIR Native Extension with Eclipse – Part 4

In this final of my 4-part video series, I show you how to import and use the ANE that we created in the last three videos.  We will be using Adobe Flash Builder 4.6 to import the ANE, and we will build a very quick sample application that will use the getTestString and getHelloWorld functions that we wrote in our native DLL written in C.

If you want a copy of all the final projects, you can download them here.  The ZIP file includes the CDT project, the compiled DLL, the ActionScript project, the compiled ANE and the project created in this fourth video.  Enjoy!

Working with AIR Native Extensions on the Mac

As I said in my previous blog post, creating the ArduinoConnector ANE for the MacOS X platform was frustrating.  It turns out that Apple ships a slightly modified version of GCC that is close to what is offered on the Linux/Unix and Windows platforms, but not 100% the same.  When I approached the project, I thought it would be a cake-walk — simply take CDT (or gcc command-line), and point it to my POSIX C code, and I should be a few clicks away from creating the .framework file needed for AIR.

Turns out that the only way to make a .framework file is by using Apple’s Xcode (unless I’m really missing something in the gcc man pages).  I wasn’t really in the mood to learn yet another coding IDE, but I did want to post some of the gotcha’s I found in creating the Mac version of the ANE.  These are not step-by-steps, but rather things that tripped me up.

  • When you create a new Xcode project, you must choose a Cocoa Framework project (under Framework & Library).
  • Add the “Adobe AIR.framework” by clicking on the project name, and then going to Add, Existing Frameworks.  The Adobe AIR Framework file is found in your “Flash Builder App Dir -> sdks -> 4.6.0 -> runtimes -> air -> mac”  folder.
  • The Adobe AIR Framework file auto-includes the header file, and the external library that you need to link to.  Don’t go searching for another header file, you don’t need it.
  • Create/Import your .c and .h files and attempt to compile before setting your project settings.  If you don’t try to compile before you set your project files, you won’t see 3/4 of the compiler settings you need to make things right.  This one threw me into a loop for at least a day.
  • All of the examples have you including the <Adobe AIR/Adobe AIR.h> file in your project — I found this works, but all it is doing is including the FlashRuntimeExtensions.h that exists within the Adobe AIR framework.  I decided to resist change and include the Adobe AIR.h file.  It works.
  • There are two places you need to set your compiler and project settings.  One is on the project itself (the top-most node in the Groups & Files tree), and the other is in the Targets -> ProjectName.framework node.  Set all settings in both places (they don’t cascade once it has been compiled once, which you have to do to get the GCC settings to show up right).
  • Make sure your project is compiling 32-bit frameworks, targeted to the 10.5 SDK.   I never found that documented anywhere, and it was sheer luck I tried those combos.
  • Make sure that you turn the optimization level to None (0).  If you don’t do this, things will seem to work, but if you use any of the FREGet or FRENew functions, they will produce seemingly random numbers (I was convinced it was passing me back pointers casted as integers, but who knows).
  • Make the following target and project settings changes that deviate from the defaults :
    • Architetures : 34-bit Universal
    • Base SDK : Mac OS X 10.5
    • Build Active Architecture Only – Checked
    • C/C++ Compiler Version : GCC 4.2
    • Linking Compatibility Version : 1
    • Linking Current Library Version : 1
    • Other Linker Flags : -flat_namespace -weak_Framework “Adobe AIR”
    • Runpath Search Paths :
      • @executable_path/../runtimes/air/mac
      • @executable_path/../Frameworks
      • /Library/Frameworks
    • Packaging Framework Version: A
    • Wrapper Extension : framework
    • Framework Search Paths :
      • /Applications/Adobe\ Flash\ Builder\ 4.6/sdks/4.6.0/runtimes/air/mac
    • Library Search Paths: /Library/Frameworks/Adobe\ AIR.framework/Versions/1.0
    • GCC 4.2, Generate Position-Dependent Code: Unchecked
    • Optimization Level : None [-O0]
    • Other C Flags : -fvisibility=hidden
    • Other C++ Flags:  -fvisibility=hidden
    • Precompile Prefix Header : checked
  • In order to export the proper function names for your initialization and finalization functions, prefix them with __attribute__((visibility(“default”)))

Once you are creating your .framework properly, you should be all set.  If you created truly POSIX or standard C code, there should be very little changes you should need to make your project work.

I do have to make one additional note about a bug people have been finding with Flash Builder 4.6, as it was released on 11/30.  If you are attempting to include an ANE into a project, the project will compile, but if you debug the project, it will complain that it can’t find the method name (not the initilizer or finalizer, but one of your other method names).  If you debug your project from ANT or the command line, it will work fine.  The work-around for this is the following steps :

  • Make sure you have ADL in your search path.  Edit the /etc/paths file and add a new line with /Applications/Adobe Flash Builder 4.6/sdks/4.6.0/bin
  • Create a new folder on your desktop, Document folder, or somewhere where you will find it.  Copy your .ANE into it.
  • Create a new folder within this folder, and name it MyTestANE.ane    (make sure to include the .ane — that is very important)
  • cd into that new .ane directory, and execute the following command :    unzip ..\ArduinoConnector.ane      (assuming you named your .ane ArduinoConnector.ane)
  • Go into the bin-debug folder of your project, and run the following command :    adl -profile extendedDesktop -extdir ~/Documents/Extensions/ Main-app.xml    assuming that the directory you created in the second step was named Extensions and was located in your Documents folder.
  • You can repeat the last step as often as you need with no additional work, assuming you don’t need to update your .ane.   If you need to attach the debugger / profiler , you can follow the steps outlined on this blog (but use about:blank instead of http://localhost)

Well, I think that is about it.  I hope I can help alleviate some of the frustration that I suffered for somebody else in working with all this :)

Connecting your Arduino to AIR using an AIR Native Extension

One of the things that has been bugging the the most when working with Arduinos and AIR is the fact that you needed some sort of proxy application running in order to allow the AIR (or Flash) application to talk to the Arduino.  You see, the Arduino only exposes a Serial port (via USB), and AIR is limited in this regard.  The way people have gotten around this so far is to have AIR create a TCP socket connection to some sort of Proxy application, which in turn opens the Serial port to the Arduino.  It works, and it works very well, however this is a separate application you have to ship, have the user configure, and run.  Heck, even me as the developer found this solution clumsy when demoing these solutions.

I finally decided to fix the problem and write an AIR Native Extension.  The AIR Native Extension (ANE) is a C based .DLL / .framework for the Windows/Mac platforms that allows AIR to essentially open a COM port.  I wrote it in a way that is supposed to emulate the functions of the flash.net.Socket library that is included in the AIR runtime.  I’ve posted the entire project, including the source code and final binaries on Google Code at http://code.google.com/p/as3-arduino-connector/ (well, everything except for my compilation scripts, which are specific to my computers). 

The biggest learning experience in creating this ANE was developing on the MacOS platform.  I’ve never done any programming targeted for that platform before, and working with XCode is just a pain.  I’m used to environments like Visual Studio and Eclipse, but XCode always seemed to fight me every step of the way.  From simple things like hiding all the project properties, to trying to force you into an MDI workflow.  Also, working with the lack of documentation on the AIR Runtime side was kind of depressing…  Don’t worry, a future blog post will try to fill everybody in on how to make an ANE using XCode.

Using the ANE :

  • Include the ANE into your project.  Make sure you are using Flash Builder 4.6 or later. Right-Click on your project in the Package Explorer (Project Explorer), and go to Properties.  Go to the Flex Build Path tab, and then the Native Extensions tab.  Click “Add ANE…” and bring it in.  It does not need to reside within your project source folder.
  • Next, import the com.quetwo.Arduino.ArduinoConnector and the com.quetwo.Arduino.ArduinoConnectorEvent packages.
  • Instantiate a new variable of type ArduinoConnector.
  • Check the isSupported() function to make sure that the ANE is supported on your specific platform, and if it loaded properly.
  • Call the getComPorts() function which will return an array of valid list of COM ports on your OS.  On Windows, this returns ALL COM ports that are valid in Windows, where on the MacOS platform, it will return any USB Serial Devices, which usually would only be your Arduino.
  • Make the connection to the Arduino by calling the connect(comPort, baud) function.  You will need to pass in one of the COM ports from the getComPorts() array, along with the baud rate that your Arduino is operating on.  If you are using Firmata, it is 57600.  Most other Arduino projects use 9600.
  • Next, add an event listener to listen for the “socketData” event.  This will fire when new data is available in the data buffer.  Don’t wait too long to pull data out of the buffer, because it is only 4k :)

Sending Data:

Sending data is just like the Socket class, except you don’t have to flush() the buffer — the data goes out in realtime.  The two most common ways to send data :

  • writeString(“hello world”);
  • writeByte(255);

Getting Data from the Buffer:

As data arrives and is placed into the buffer, the bytesAvailable variable is incremented to reflect how many bytes are available.  Most people will read data from the buffer using one of the two functions :

  • readBytesAsString();
  • readByte();

All of the read functions are FIFO, meaning they will return the oldest data in the buffer before they return the newest (First In, First Out).

On the Google Code site, I am posting more detailed documentation, including a simple patch to as3Glue which will allow as3Glue to work seamlessly with this ANE. 

Enjoy, and please provide feedback on this ANE.  It’s the first one I’ve released to the public and I’d like to know how it works for everybody!

Creating a Windows AIR Native Extension with Eclipse – Part 3

In part 3 of this video series, I talk about how to write the AS3 portion of the AIR Native Extension, and how to package all the components into your final .ANE file. 

You can watch part one here, and part two here.  Part four will talk about how to use the .ANE within Flash Builder.  I will post all the files used in this tutorial in the final blog post.

Creating a Windows AIR Native Extension with Eclipse – Part 2

In part two of this video series, I go through how to actually program your ANE Windows DLL.  This involves doing some C programming.  Please see part 1 here.

The snippets mentioned in this video are available here :  ANE Snippets Download     You can use these to jump-start your development.

Creating a Windows AIR Native Extension with Eclipse – Part 1

The second I heard about Adobe giving us the ability to create our own extensions to the Flash Platform in AIR 3.0, I was smitten.  It was finally a way that we could add our own features and do the things that were high priorities on our lists, but not on Adobe’s.  I knew I was looking for features that were one-offs (how many people today really need access to the COM ports), but they were forcing me to do all sorts of weird workarounds like launching proxy applications to do seemingly simple tasks.

AIR 3.0 got released a few weeks ago and I’ve jumped in head first into creating some ANEs (AIR Native Extensions).  For those of you who don’t know, ANEs are packaged extensions that contain operating-system specific code (DLLs for Windows, Libraries for MacOS, Java classes for Android and Objective-C for iOS), that allow you to do things that the Flash Player wasn’t able to do. 

Unfortunately, Adobe assumed that if you were developing DLLs for Windows, you were going to be using Visual Studio and nothing more.  This didn’t make a whole lot of sense in my mind as they’ve been leveraging Eclipse for all of their tooling, and Eclipse does offer some great C/C++ addins.  Now, that being said, Visual Studio is by far the more feature-full and hands-down the best editor for enabling these kinds of workflows on Windows.  It is, however, very costly and even though Microsoft offers a free versions, it takes over your computer by installing debug versions of most of Microsoft’s shared libraries making your computer slower and more crash prone.

I wanted to use Eclipse’s CDT addin with the standard GCC tooling that is available on pretty much every operating system.  By using GCC, I was able to make very portable code that with minimal effort was able to compile on all three of the major OSs (Windows, Mac, Linux). Adobe’s documentation was little help in getting this going (even if you were coding in Visual Studio, there is very little guidance on how to get things setup).  I do have to note that with my setup there is one distinct disadvantage — the lack of ability to debug the DLL when it is launched from AIR.  You will have to write your own C/C++ harness to do testing on your code in order to test it.  If you use the Visual Studio tooling, you CAN debug any DLL while it is running (this is why Microsoft replaces the shared libraries on your system to allow that debugging).

I’ve created a four part video series documenting how to get going creating ANEs.  Part 1 covers setting up your environment, including installing CDT, the compiler, and getting Eclipse setup to do your programming.  Part 2 covers actually coding the C/C++ code for your Windows DLL.  Part 3 covers creating your ANE, and packing up all the stuff needed to make it work.  And Part 4 covers how to use your new ANE in an AIR project.

My Presentations at Adobe MAX 2011

This year I had the distinct honor of being asked to present at Adobe’s MAX conference.  The conference was an absolute blast.  From the Keynotes to ALL the other sessions I attended, the thing went off without a hitch.

I gave two presentations this year — one for the 360|MAX Unconference and one for the Develop Track at MAX. 

Getting Data From Here To There (with Flex)

This session was not recorded, but fairly well attended.  You can see my slides here.   In this session I talked about the different communications methods available to Flex developers, and I started to lay out a basic matrix of when to use what type of communication method, and what the pros and cons were of each type.  Not all of my demos worked due to a broken J2EE server, but I think everybody got the idea.  I don’t have great downloads for that presentation as most were specific to my server setups.

Getting Physical With Flash (Hardware Hacking)

This session was a blast to present.  We had about 140 people in the room who seemed to be really into it.  I presented on integrating the Arduino hardware prototyping kit into Flash/Flex in addition to showing how to integrate the Microsoft Kinect into Flash/Flex. I came armed with about 6 electronics projects that I showed people to inspire them to create their own.

Video Player - MAX 2011 Preso

You can download the PPT here.

You can find most of the downloads featured on my blog, but I will update this post and post the direct links to everything at a later date.

Thanks again to the entire MAX staff for making the show run so smoothly from both the speaker’s perspective and from the attendee’s perspective.  A+ work led to an A+ experience :)

Integrating the Microsoft Kinect into your Flex application

One of the topics I will be talking about during my MAX session this year is integrating game controllers into your Flex/AIR or ActionScript games in order to make more naturally interactive games.  There are a lot of examples of using the Wii controller online already — and plus with the next generation of video game consoles coming out soon, I thought it would be best to spend my time on the latest and greatest — the Microsoft Kinect.

For those who don’t know much about the Kinect, it is a pair of Web Cams — one RGB (like a normal Web Cam), and one that sees only infrared.  The combination of the two (along with a bunch of other electronics and software) allow the Kinect to have real depth-preception.  This means that being able to track people, hands and everything else become trivial.  The Microsoft Kinect was actually developed by PrimeSense, who originally made the device for PCs, rather than game consoles.  This means that the devices are actually compatible with PCs, rather than relying on a whole stack of hacks to get them to work. 

The biggest difficulty in getting the Kinect working is wading through all the pieces and parts that are needed to get it all working.  Documentation is in very short supply (and usually found within text files on GitHub).  Compounding the problem is that there are tens, if not hundreds of open-source projects revolving the Kinect — but unfortunately each depend on a different set of drivers, middleware and APIs — many which are NOT compatible with eachother. 

I will state right here that I am not an expert in all of these, but I did follow one path that made things work (after following MANY that didn’t).  Matt LeGrand and I are planning a separate website and blog that will explore these other paths more to help people make sense of what to choose and what not to choose.  My primary experience is getting the Kinect working with Windows 7, as I am well aware that my aging Mac laptop is not powerful enough to do much with the Kinect.  Yes, you need a modern computer in order to use the Kinect.

So, where would you start?  You obviously need the actual Microsoft Kinect controller.  There are two models in the market right now — one that is USB only, and one with a USB + Power supply.  The USB only version will ONLY work with the XBox, where the USB + Power Supply version works with PCs (the Kinect draws a few amp of power, which the XBox apparently can supply using USB).  The only difference between the two packages is the power brick and a USB power injection module.  If you end up with the wrong one, you are supposed to be able to buy the power brick and injection module, but I have no idea where one would pick that up.  I got my kit from Best Buy in the video game refurbished aisle (it was about $100, after an instant coupon).  The brand-new ones were about $140.

Before you plug it in, you need to install the drivers, middleware and APIs.  There are three well-known sets of drivers and middleware, but the ones I found worked for me are directly from PrimeSence.  The drivers and middleware are hosted on OpenNI’s website at http://www.openni.org/downloadfiles/opennimodules.  There are three downloads you need — the OpenNI Module (OpenNI Binaries), the PrimeSense NITE Middleware, and the PrimeSense Drivers (OpenNI Compliant Hardware Binaries).  Download only the 32-bit downloads, even if you are on a 64-bit Windows 7 install!!  I’ve found LOTS of issues with the 64-bit drivers that cause things to break.    Install the drivers, then the middleware, then the OpenNI APIs (in that order).

Finally, you will need the as3 modules.  There is an open-source project known as AS3OpenNI that makes programming towards the OpenNI APIs very simple.  Because AIR can’t directly talk to the APIs you have to use the included C++ application that proxies the driver calls to a TCP/IP connection.  I’m sure this will become easier in future versions of AIR.   AS3OpenNI takes all the hard word of processing the blob data that comes back from OpenNI and gives you either skeletal data (as a class), RGB, Depth, or “multitouch blob” data.  With this data you can read back data from multiple users and track their hands, head, neck, legs, etc.  I built all of my stuff on version 1.3.0, which is really stable.

Take a look at the examples in the AS3OpenNI project — they are pretty descriptive, and once you get all the parts working together, they work really, really well :) 

So, what did I make using the Kinect?  One of my first games I threw together was a simple version of Space Invaders.  I have AS3OpenNI track the right hand to move the player left and right, and when the right hand moves over the center point of the neck, I fire a missile.  The following code is all that is really needed (minus the setup of the player, etc.) :

protected function gotShipMove(event:ONISkeletonEvent):void
{
 var rightHand3D:NiPoint3D = event.rightHand; // get the right hand's x,y,z
 var rightHand:NiPoint2D = NiPoint3DUtil.convertRealWorldToScreen(rightHand3D, this.stage.width, this.stage.height);
    
 shipIcon.x = rightHand.pointX;
 if ((event.skeleton.rightHand.pointY > event.skeleton.neck.pointY) && canFire)
 {
  if (!fireInProgress)
  {
   fireInProgress = true;   // prevent 5,000 missiles from firing at once...
   var missile:MissileItem = new MissileItem(spaceInvadersCluster);
   missile.x = rightHand.pointX;
   missile.y = height - 64;
   missile.addEventListener("MissileFireComplete", missileFireComplete);
   missile.addEventListener("MissileFireHit", missileFireComplete);
   addElement(missile);
  }
 }
 else
 {
  fireInProgress = false;
 }
}

Watch my twitter stream for more information about our (Matt and I) new website dedicated to the Kinect and AS3 :)  I will also cover the game a lot more during MAX, so make sure to come!

The BikePOV. Adobe AIR + Arduino + Blinking lights on a bike

So, for the past month I have been working on a side project called the BikePOV.  If you have been reading my tweets, I’m sure you’ve picked up on my cursing, explaining and working on making it work. 

This evening I finally got everything working just the right way — and it actually works!

So, first let me explain what is going on.  I took an Arduino prototyping board and designed a circuit around it.  Essentially I took 12 RGB (Red, Green, Blue) LEDS and soldered them onto a circuit board.  I then mounted the circuit board in between the spokes of a bike wheel.  The theory is that when the wheel turns, I can control the LEDs, and make them flash in a pattern that represents letters, patterns or images.  This is called a POV, or Persistance of Vision. 

This idea has been done before — there are pre-made kits that you can buy from a company called AdaFruit.  A company called Monkeyletric also sells a POV kit for about $60 (which is MUCH nicer than my setup, but they only have pre-done patterns). Read more of this post

Follow

Get every new post delivered to your Inbox.

Join 28 other followers