Projection Manipulation With Maxedia

Jun 9, the defense is over, it can be considered a successful one.  Now the thesis is really behind me, however, other “business” has to be taken care of. I'm looking forward to a break.

The slides:
users.groept.be/bo/bo.gao/site/Final/Thesis Presentation.ppt

Thesis was handed in on May 12, as in the paper,  the appendices and the holistic context were finished, although the content on the blog is not complete.

everything is here
users.groept.be/bo/bo.gao/site/Final/final handin.zip

So far, all the lines or curves on the meshes are actually drawn by GDI, i.e. I directly use Graphics.DrawLine or Graphics.DrawBezier, whose advantage is Hardware Acceleration, but the disadvantage is that we don't have  the intersection points among all the lines or curves, which I have to calculate by myself.

The intersection of lines is easy to derive, but the intersection of two cubic bezier curves is a trouble. I finally chose Bezier Clipping to solve this intersection problem.

The concept of Fat Line:


A cubic bezier curve with four control points: P0,  P1,  P2 and  P3, the middle red line connects P0 and P3, the blue lines which are parallel with the middle red line are the fat line, which try to enclose the curve as close as possible, theoretically, the fat lines are supposed to be tangent with the curve.  The shade indicates the enclosed area of fat line.

Bezier Clipping:


The Fat LInes of one bezier curve cut  the other bezier curve into a smaller piece of sub-bezier curve highlighted in yellow. This smaller piece of curve then uses its own fat lines to cut its partner, so on and so on, iteratively untill we got  a sufficiently small  piece of bezier curve which is then considered to be the intersecting point of the two original bezier curves.

The ultimate goal of the bezier patching is the same as the perspective one, the user should be able to drag the controls to transform the mesh in a 3D space, although in our case, we are as innocent as the user, we don't have the depth information either... so let's fake a 3D looking mesh!

A lame idea -  I gave the timelines(the parameter t in the equation) of the curves on the mesh some perspective behavior based on the perspective calculations I've done before. It turned out that a perspective timeline didn't give us a perspective (in other words - rational) bezier curve.  But in the sense of intuitively helping the user, it worked.



All the source code and dll's for both perspective and bezier patching video demo:
users.groept.be/bo/bo.gao/site/VideoRenderingDemo/VideoRendering.zip

Next step is to make a curved screen with the help of bezier patching so that the end user not only can transform the image perspectively but also really create richer shapes according to the chaging environment.

Here is the approach:


Prerequisites: the Bezier curve and Bezier patch we are talking about are all cubic, i.e. we conform with this equation:
B(t) = (1-t)^3 * P0 + 3 * (1-t)^2 * t * P1 + 3 * (1-t) * t^2* P2 + t^3 *P3.

1. Each bezier patch consists of 16 control points which allow user to drag(of course when multiple patches are connected to comprise a bigger mesh, there can be too many controls, then we need to eliminate some of them, but that's a disscusion for later).Then 24 straight lines connecting all these 16 control points, we got a so-called backbone of the patch.
2. In order to interpolate with the intermediate bezier curves, we need to find the intermediate control points, a method I found useful is to find the 8 bezier curves connecting the backbone which I call "the muscle"(as the red curves indicate), and I choose those intermediate controls from "the muscle".(One thing I have to clearify is that this is a pure 2D trick to play with, there's no 3D meaning behind it...)
3. Once we got all the controls, we are ready to interpolate  with bezier curves, the horizontal muscles provide cotnrols to vertical curves, the vertical muscles provide controls to horizontal curves.

——  Perspective Implementation (Generic , video testing)

Only for testing, the communication between the vertex array containing the position and texture uv coordinates which is generated from the GUI mesh application and the run time video rendering is done by xml, a slow one and not simultaneously.
So those subtle improvements will be considered later.

As always, here are the screen prints and the code:

 

 

 

The generic perspective library: 
1) line:users.groept.be/bo/bo.gao/site/Generic Perspective/Lib&GUI/Line.cs
2) mesh: users.groept.be/bo/bo.gao/site/Generic Perspective/Lib&GUI/Mesh.cs
The GUI application: testApp:
users.groept.be/bo/bo.gao/site/Generic Perspective/Lib&GUI/TestApp.zip
The video rendering:
users.groept.be/bo/bo.gao/site/Generic Perspective/videoRender/videoRenderExample.zip

       

In order to draw the mesh with different numbers of squares on width and height, Jan and I came up with a new algorithm using what I call the "magic line".
The magic line connects two vanishing points, then all we have to do is to find another line with the same slope of the magic line that goes through the mesh, intersecting with the four defining lines of the vanishing points,
presently deriving the four intersecting points which compose two sections.
Followed by the equipartitions of the two sections  and with the help of vanishing points, we can cut up the mesh in whatever width and height compositions as we want!

In spectial situations such as two out of four side lines are parallel with each other, only one vanishing point exsits. When this happens, we can equally cut the side line and intersect one diagonal of the mesh to find what I call the "communication points" through which the parallel lines that cut the mesh in the other dimention are found.

What if the mesh becomes a parallelogram? Tha's the simplest case, dividing the mesh equally on the width and height respectively.

The beauty of this algorithm is that it enables us to generically define whatsoever mesh we want, i.e. the density of the mesh and the width/ height ratio can be arbitrarily specified.

——  Perspective Implementation (hard code)

Points are rigid, not flexible for generally defining a mesh. 

 
                  

The source code (in C#) can be downloaded here:
Perspective Library: users.groept.be/bo/bo.gao/site/lib/PerspectiveMesh.cs
The Form: users.groept.be/bo/bo.gao/site/form/MarkControl_v1.cs
                      users.groept.be/bo/bo.gao/site/form/test_v1.cs

The perspective transformation:


The horizontal Bezier transformation:


The vertical Bezier transformation:


The full Bezier transformation:


The software "WATCHOUT" can be downloaded in this page: www.dataton.com/#/downloads/;
Dataton's website:  www.dataton.com/;

PS: This is not an advertisment, and I can build a better software  regarding the aspect of Intuitive design, but thanks a million to the forthgoers!

Project Proposal:


plan of approach:

PROJECTION MANIPULATION WITH MAXEDIA

Statement of Problem

         Maxedia is a graphic server that generates light effects in real time. The video material can easily be changed in color or combined with live effects.  Everything is processed and real-time rendered as input to for projectors, plasma screens or video walls.
The problem with projecting on walls or similar screens is that they can be bent, curved, tilted or a combination of the previous. This will result in a distortion off your projection as it is perceived by the audience.
Maxedia doesn’t have a tool to transform the projected image to make it conform to the particular shape of the screen.

 

Design Objectives

 

            This document proposes that we will solve the problem stated above in five basic objectives which we will shortly explain here.

1.      Literature Study

2.      Mathematical calculations, implementations and test

3.      An test application in C#

4.      Integrate the mathematical implantation into Maxedia with C++

5.      Build the GUI of the new Maxedia functionality.

·          The literature study is for us to make ourselves more tuned with the concepts of computer graphics, perspective and the math’s behind is. We will also try to get a better understanding of C# and C++.

·        The mathematical calculation is to find the physical mathematical rules for perspective viewing. We will do this to understand the mathematics behind the media that we will have to process later.
When we know how this math works we will implement is into a C# class library that can work independently and of course test its correctness.

·        According to the mathematics we will make an application to test everything in a higher level with a user interface on which we can transform a virtual screen intuitively.

·        If the third step has been proved to work we can start integrating everything into the Maxedia with C++ so that we will generate an output-mesh that can be used for practical purposes like video-processing.

·        Of course everything needs a friendly and good-looking user interface that allows users to intuitively manipulate the projection-image to create the exact mesh. Of course this interface has to blend in perfectly with the rest of the already existing Maxedia UI. 

 

Technical Approach

            See WBS

 

Identifying Customer Needs

            The costumers need a nice and easy way to transform the mesh of the projection in more ways.

 

Generating Design Concepts

            Generally, the design concept is to achieve a 3D visual effect with a 2D mathematical implementation.

            This mathematical implementation includes Perspective Calculation and Bezier Patching. Perspective calculation gives the rotation and scaling effect, while the Bezier patching makes the projected image curvy like. When the perspective calculation and Bezier patching are combined together, the user can manipulate the projected image into an arbitrary shape with arbitrary angle.

            Then we make these mathematical implementations into libraries which the programmer can refer to very easily later.

            When we have derived these libraries, we test them, prove them and we plug them into Maxedia environment to enrich the functions of it, make it meet customers’ needs in a better way.

 

Selecting Design Concept

            We choose the Bezier-curves and patching to simulate the curving on a screen because it provides a nice compromise between usability and programmability.

            Also we simulate a 3D effect in 2 with perspective transformations, because in a screen 2D is more intuitively to manipulate then a 3D scale of the screens.

 

Project Management

See Gantt-chart

 

Deliverables

            We hope to create a nice working plug-in for the Maxedia media server which extends its use the things we’ve just explain in the previous paragraphs.

            To achieve this, our first deliverable is a class library for the mathematics.
The second one is a working graphic prototype on which the end user can manipulate a mesh.

       And last we will make a plug-in based on the knowledge we’ve gathered from creating the test-GUI and the mathematics.


WBS. v2:



Gantt Chart. v2: