After cranking out a post per day so far this week on my exploratory integration of AutoCAD with the Leap Motion controller, it’s time to wrap up the technical portion of my “Leap Week” (a bit like a Leap Year, geddit? ;-) with a nice, juicy topic: creating 3D geometry inside AutoCAD using the Leap Motion controller.
The Leap Motion controller has a couple of key selling points for interacting in 3D space. It’s both highly reactive – they’ve done a great job of minimising any processing lag to allow you to build highly responsive systems – and very accurate.
The accuracy bit makes me think back to when we last had “accurate” input from a peripheral device, when people used to digitise 2D using a digitising tablet. This process gradually died out after the introduction of the mouse: a less precise way of working, but perfectly acceptable when used in conjunction with drawing aids such as grid and object snapping.
The Leap Motion controller is indeed very accurate, but – aside from its application for navigation, etc. – I think it’ll primarily be of interest to people doing rough, conceptual 3D sketching and sculpting rather than modeling precisely in 3D. The accuracy is there, but it’s just really hard to keep your hand steady enough to make use of it. You can do things to mitigate this (beyond not drinking too much the night before ;-), such as having your code only accept points that are a certain distance from the last one. While this can help cut out a fair amount of “jitter”, I don’t personally feel that the device’s accuracy is going to be a compelling selling point for people who really care about precision.
On to the implementation, then…
I had a bit of a head-start on coding this one, as I’d used a similar approach when creating 3D geometry based on input from Kinect. I created a new Jig for this, integrating the navigation code from Tuesday’s post into it.
A cursor – a sphere – will appear at the tip of the primary “pointable” (which could be a tool or a finger) when either just one or two get detected. There’s a little bit of magic involved in transforming the coordinates coming from the Leap Motion controller to make sure they more-or-less match up with the current view: it’s a little strange to be modeling in 3D when the coordinates are inverted, for instance.
The user can then use the Jig’s keyword input to launch the creation of a spline or 3D polyline based on the movements of this cursor, terminating with the Enter key. Navigation can occur at any point during the command, of course.
Here’s the C# code, which essentially replaces that posted on Tuesday (and can still be used in conjunction with the code from Wednesday).
using System;
using System.Threading;
using System.Collections.Generic;
using Autodesk.AutoCAD.ApplicationServices;
using Autodesk.AutoCAD.ApplicationServices.Core;
using Autodesk.AutoCAD.Colors;
using Autodesk.AutoCAD.DatabaseServices;
using Autodesk.AutoCAD.EditorInput;
using Autodesk.AutoCAD.Geometry;
using Autodesk.AutoCAD.GraphicsInterface;
using Autodesk.AutoCAD.Runtime;
using Leap;
namespace LeapMotionIntegration
{
class JigUtils
{
// Custom ArcTangent method, as the Math.Atan
// doesn't handle specific cases
public static double Atan(double y, double x)
{
if (x > 0)
return Math.Atan(y / x);
else if (x < 0)
return Math.Atan(y / x) - Math.PI;
else // x == 0
{
if (y > 0)
return Math.PI;
else if (y < 0)
return -Math.PI;
else // if (y == 0) theta is undefined
return 0.0;
}
}
// Computes Angle between current direction
// (vector from last vertex to current vertex)
// and the last pline segment
public static double ComputeAngle(
Vector3d dir, Vector3d xdir, Matrix3d ucs
)
{
var v = dir / 2;
var cos = v.DotProduct(xdir);
var sin =
v.DotProduct(
Vector3d.ZAxis.TransformBy(ucs).CrossProduct(xdir)
);
return Atan(sin, cos);
}
}
public class LeapJig : DrawJig
{
// Internal state
private Document _doc;
private Point3dCollection _verts;
private Point3d _pos;
private bool _enterPressed;
private bool _showCursor;
private bool _drawing;
private bool _splineVsPline;
public LeapJig(Document doc)
{
// Initialise the various members
_doc = doc;
_verts = new Point3dCollection();
_pos = Point3d.Origin;
_enterPressed = false;
_showCursor = false;
_drawing = false;
_splineVsPline = false;
}
public Point3dCollection Vertices
{
get { return _verts; }
set { _verts = value; }
}
public Point3d Position
{
get { return _pos; }
set { _pos = value; }
}
public bool EnterPressed
{
get { return _enterPressed; }
set { _enterPressed = value; }
}
public bool ShowCursor
{
get { return _showCursor; }
set { _showCursor = value; }
}
public bool Drawing
{
get { return _drawing; }
set { _drawing = value; }
}
public bool SplineVsPline
{
get { return _splineVsPline; }
set { _splineVsPline = value; }
}
protected override SamplerStatus Sampler(JigPrompts prompts)
{
// We don't really need a point, but we do need some
// user input event to allow us to loop, processing
// for the Leap Motion input
var opts =
new JigPromptPointOptions(
"\nEnter to commit polyline/reset view, esc to quit " +
"or [Pline/Spline]",
"Pline Spline"
);
opts.UserInputControls =
UserInputControls.NullResponseAccepted;
var pr = prompts.AcquirePoint(opts);
switch (pr.Status)
{
case PromptStatus.Keyword:
{
_splineVsPline = (pr.StringResult == "Spline");
_drawing = true;
break;
}
case PromptStatus.None:
{
_enterPressed = true;
return SamplerStatus.OK;
}
case PromptStatus.OK:
{
ForceMessage();
return SamplerStatus.OK;
}
}
return SamplerStatus.Cancel;
}
protected override bool WorldDraw(WorldDraw wd)
{
try
{
var vtr = _doc.Editor.GetCurrentView();
var hgt = vtr.Height;
var wid = vtr.Width;
var min = hgt < wid ? hgt : wid;
var fac = min / 300;
var pt = _pos * fac;
var curRad = hgt / 70;
// Get the displacement to the specified position
var mat = Matrix3d.Displacement(pt.GetAsVector());
// Apply the view's rotation, so we compensate
var rot =
JigUtils.ComputeAngle(
vtr.ViewDirection,
Vector3d.YAxis,
_doc.Editor.CurrentUserCoordinateSystem
);
var rotMat =
Matrix3d.Rotation(
rot + Math.PI, Vector3d.ZAxis, Point3d.Origin
);
mat = mat.PreMultiplyBy(rotMat);
// Create and draw our solid
if (_showCursor)
{
using (var cursor = new Solid3d())
{
cursor.CreateSphere(curRad);
cursor.ColorIndex = 3;
cursor.TransformBy(mat);
cursor.WorldDraw(wd);
}
}
// If drawing, add vertices when at a reasonable distance
// from the previous one
if (_drawing)
{
if (
_verts.Count == 0 ||
_verts[_verts.Count - 1].DistanceTo(pt) > curRad * 5
)
{
_verts.Add(pt.TransformBy(rotMat));
}
}
else if (_verts.Count > 0)
{
// If not drawing and there are vertices, clear them
_verts.Clear();
}
// If we still have vertices in the list...
if (_verts.Count > 1)
{
// Create a polyline or spline and draw it
using (var ent = GeneratePathEntity())
{
ent.ColorIndex = 2;
ent.WorldDraw(wd);
}
}
}
catch { }
return true;
}
public Entity GeneratePathEntity()
{
return
(_splineVsPline ?
(Entity)new Spline(_verts, 0, 0.0) :
new Polyline3d(Poly3dType.SimplePoly, _verts, false)
);
}
private void ForceMessage()
{
// Set the cursor without ectually moving it - enough to
// generate a Windows message
var pt = System.Windows.Forms.Cursor.Position;
System.Windows.Forms.Cursor.Position =
new System.Drawing.Point(pt.X, pt.Y);
}
}
public class Camera
{
// Members
private Document _doc = null;
private ViewTableRecord _vtr = null;
private ViewTableRecord _initial = null;
public Camera(Document doc)
{
_doc = doc;
_initial = doc.Editor.GetCurrentView();
_vtr = (ViewTableRecord)_initial.Clone();
}
// Reset to the initial view
public void Reset()
{
_doc.Editor.SetCurrentView(_initial);
if (_vtr != null)
{
_vtr.Dispose();
_vtr = (ViewTableRecord)_initial.Clone();
}
_doc.Editor.Regen();
}
// Zoom in or out
public void Zoom(double factor)
{
// Adjust the ViewTableRecord
_vtr.Height *= factor;
_vtr.Width *= factor;
// Set it as the current view
_doc.Editor.SetCurrentView(_vtr);
// Zoom requires a regen for the gizmos to update
_doc.Editor.Regen();
}
// Pan in the specified direction
public void Pan(double leftRight, double upDown)
{
// Adjust the ViewTableRecord
_vtr.CenterPoint =
_vtr.CenterPoint +
new Vector2d(
leftRight * _vtr.Width,
upDown * _vtr.Height
);
// Set it as the current view
_doc.Editor.SetCurrentView(_vtr);
}
// Orbit by angle around axis
public void Orbit(Vector3d axis, double angle)
{
// Adjust the ViewTableRecord
_vtr.ViewDirection =
_vtr.ViewDirection.TransformBy(
Matrix3d.Rotation(angle, axis, Point3d.Origin)
);
// Set it as the current view
_doc.Editor.SetCurrentView(_vtr);
}
public void OrbitVertical(double angle)
{
// Adjust the ViewTableRecord
_vtr.ViewDirection =
_vtr.ViewDirection.TransformBy(
Matrix3d.Rotation(
angle,
_vtr.ViewDirection.GetPerpendicularVector(),
Point3d.Origin
)
);
// Set it as the current view
_doc.Editor.SetCurrentView(_vtr);
}
}
public class NavigationListener : Listener
{
private Editor _ed;
private Camera _cam;
private SynchronizationContext _ctxt;
private LeapJig _jig;
public NavigationListener(
Editor ed, Camera cam, SynchronizationContext ctxt, LeapJig jl
)
{
_ed = ed;
_cam = cam;
_ctxt = ctxt;
_jig = jl;
}
private void WriteLine(String line)
{
if (_ed != null)
{
_ctxt.Post(a => { _ed.WriteMessage(line + "\n"); }, null);
}
}
private void CallOnCam(SendOrPostCallback cb)
{
if (_cam == null)
WriteLine("Cam is null.");
else
_ctxt.Post(cb, null);
}
private void Reset()
{
CallOnCam(a => { _cam.Reset(); } );
}
private void Zoom(double factor)
{
CallOnCam(a => { _cam.Zoom(factor); });
}
private void Pan(double leftRight, double upDown)
{
CallOnCam(a => { _cam.Pan(leftRight, upDown); });
}
private void Orbit(Vector3d axis, double angle)
{
CallOnCam(a => { _cam.Orbit(axis, angle); });
}
private void OrbitVertical(double angle)
{
CallOnCam(a => { _cam.OrbitVertical(angle); });
}
public override void OnInit(Controller controller)
{
//WriteLine("\nInitialized");
}
public override void OnConnect(Controller controller)
{
//WriteLine("Connected");
}
public override void OnDisconnect(Controller controller)
{
WriteLine("Disconnected");
}
public override void OnFrame(Controller controller)
{
// Get the most recent frame
var frame = controller.Frame();
var hands = frame.Hands;
var numHands = hands.Count;
// Only proceed if we have at least one hand
if (numHands >= 1)
{
// Get the first hand and its velocity to check for
// zoom or pan
var hand = hands[0];
var handVel = hand.PalmVelocity;
if (handVel == null)
handVel = new Vector(0, 0, 0);
// Check if the hand has any fingers
var fingers = hand.Fingers;
var ptrs = frame.Pointables;
// Only proceed if we see at least two fingers detected
if (fingers.Count == 0 && ptrs.Count == 0)
{
_jig.ShowCursor = false;
}
else if (fingers.Count > 1)
{
_jig.ShowCursor = false;
// Set a flag to see whether we detect zoom or pan
bool zoomOrPan = false;
// Create an AutoCAD vector from the hand's velocity
var handVec =
new Vector3d(handVel.x, handVel.y, handVel.z);
// Get the largest element and its [absolute] value
var largestDim = handVec.LargestElement;
var largestVal = handVec[largestDim];
var largestAbs = Math.Abs(largestVal);
// Depending on the largest value we know to zoom or pan
switch (largestDim)
{
case 1:
if (largestAbs > 250)
{
if (frame.Id % 2 == 0)
{
WriteLine(
"Zoom " + (largestVal < 0 ? "in" : "out")
);
Zoom(largestVal > 0 ? 1.1 : 0.9);
}
zoomOrPan = true;
}
break;
default:
if (largestAbs > 100)
{
var x = 0.02 * -handVel.x / largestAbs;
var y = 0.02 * handVel.z / largestAbs;
WriteLine(
"Pan " +
(largestDim == 0 ?
(largestVal < 0 ? "left" : "right") :
(largestVal < 0 ? "up" : "down")
)
);
Pan(x, y);
zoomOrPan = true;
}
break;
}
// If not zoom or pan, we check for orbit
if (!zoomOrPan && largestAbs < 100)
{
// To determine whether we have to orbit, get the normal
// to the hand's palm
var handNorm = hand.PalmNormal;
if (Math.Abs(handNorm.Roll) > 0.4)
{
// Orbit left or right when there is "roll"
WriteLine(
"Orbit " + (handNorm.Roll < 0 ? "right" : "left")
);
var zAxis =
_ed.CurrentUserCoordinateSystem.CoordinateSystem3d.
Zaxis;
Orbit(zAxis, 0.5 * handNorm.Roll * Math.PI / 12);
}
else if (handNorm.Pitch < -1.5 || handNorm.Pitch > -1.0)
{
// Orbit up or down when there is "pitch"
var pitch =
handNorm.Pitch < -1.5 ?
handNorm.Pitch + 1.5 :
Math.Abs(handNorm.Pitch + 1.0);
WriteLine(
"Orbit " + (pitch < 0 ? "up" : "down")
);
OrbitVertical(0.5 * pitch * Math.PI / 12);
}
}
}
else
{
if (ptrs.Count > 0)
{
// Set the cursor position to be the
// tip of the primary pointable
var tip = ptrs[0].TipPosition;
_jig.Position = new Point3d(tip.x, -tip.z, tip.y);
_jig.ShowCursor = true;
}
}
}
}
}
public class NavigationCommands
{
[CommandMethod("LEAP")]
public void LeapMotionNavigation()
{
// Cancel out of any existing geometry creation
GeometryCommands.LeapMotionGeometryCreationCancel();
var doc =
Application.DocumentManager.MdiActiveDocument;
var db = doc.Database;
var ed = doc.Editor;
using (var tr = doc.TransactionManager.StartTransaction())
{
var ms =
(BlockTableRecord)tr.GetObject(
db.CurrentSpaceId, OpenMode.ForWrite
);
// Create our jig
var lj = new LeapJig(doc);
// Create a camera for our current document to adjust
// the view
var cam = new Camera(doc);
// Creating a blank form makes sure the SyncContext is
// set properly for this thread
using (var f1 = new Form1())
{
var ctxt = SynchronizationContext.Current;
try
{
if (ctxt == null)
{
throw
new System.Exception(
"Current sync context is null."
);
}
// Create our navigation listener to receive events
var listener = new NavigationListener(ed, cam, ctxt, lj);
using (listener)
{
if (listener == null)
{
throw
new System.Exception(
"Could not create listener."
);
}
// Use the listener to create the Leap Motion
// controller
using (var controller = new Controller(listener))
{
if (controller == null)
{
throw
new System.Exception(
"Could not create controller."
);
}
// Run the jig
PromptResult pr;
do
{
pr = ed.Drag(lj);
if (lj.EnterPressed)
{
if (lj.Drawing)
{
var ent = lj.GeneratePathEntity();
ent.ColorIndex = 1;
ms.AppendEntity(ent);
tr.AddNewlyCreatedDBObject(ent, true);
lj.Vertices.Clear();
lj.Drawing = false;
}
else
{
cam.Reset();
}
lj.EnterPressed = false;
}
} while (pr.Status != PromptStatus.Cancel);
}
}
tr.Commit();
}
catch (System.Exception ex)
{
ed.WriteMessage("\nException: {0}", ex.Message);
}
}
}
}
}
}
In the below demo video I’ve used a finger rather than a pointing tool (such as a chopstick), as that allows me to switch between model creation and navigation seamlessly. But the code will work very well with a chopstick, too. :-)
That’s it for my current technical investigations into the possibilities for the Leap Motion device. At some point I’d like to see whether some kind of geometry manipulation is possible, but that’s a tougher nut to crack, and for another time.
To wrap up “Leap Week”, I’ll post again tomorrow to summarise my thoughts on this interesting technology and where I find it applicable to our industry.
