Adding GPS situational awareness to full motion video.

There is software available, free according to their website, That allows you to take any video that has GPS metadata embedded and display the location of where the video was recorded on google maps. The company Remote GeoSystmes Inc. calls it "moving -Track" and it brings a heighten level of situational awareness that was not easily available to commercial drone operations. Benefits of using this software include; moving target tracking, emergency situations location services,  extreme sport track map visualizations and anything else that the location of the flight or target would be benefit from being able to conceptualize on a map background. The idea of taking coordinates of what your UAS video is looking at and being able to put a map location to it is an invaluable resource for many applications. Not many of use can read GPS data and then mental depict its location, this software does that for you. 

The additional use of MISB, FMV, files allows the software the additional layer of adding the sensor footprint, where the camera is looking, in additional to the cameras physical location to the feedback. In the first image below the red line represents the the aircraft flight track, where the green dot is the current location. The second image depicts where the actual image is in relation to the surface of the earth.

Drone Amphitheater Applied Research Capstone Project (DAARC)

Understanding Key UAS Fundamentals through Applied Research

For the successful application of UAS towards any project, key fundamentals must be understood and deployed strategically for safe end profitable operation.  Most of what is needed to fly a drone is built in to the software that automates the flight for you. This automation technology is at a point where the operation of a UAS is as simple as "point and click". However, to get the most out of a drone operations and to remain within the legal and moral boundaries of drone use, UAS fundamentals must be understood. To reach this goal, a culmination effort from eight undergraduates and one professor will be undertaken that will end with an understanding of key UAS fundamentals through applied research. The proposed Drone Amphitheater Applied Research Capstone project (DAARC) will demonstrate fundamentals of UAS applications by applying research techniques with the  creation of GIS driven products. The products will be the basis of research for a multifaceted combined research project.

Purpose
Moving forward with any project first requires purpose and direction. This "capstone" project is no different. The purpose; create a digital profile for the Tippecanoe County Amphitheater. This database would provide graphical, statistical and analytical information for future research. The hope is for the use of this data would lead to the creation of geospatial and orthomosaic maps. The initial target areas for research include flood stage analysis, man-made structure analysis, and terrain mapping. This will be done through the collection of data primary using sophisticated cameras and sensors utilizing multiple dynamic UAS platforms.

Tippecanoe County Amphitheater

Direction
Organisation will play a major role throughout the DAARC project. In order to finish the project in the amount of time available, efficiency and utilization of a timeline is required. In addition, efficiency of data collection and associated processing will be paramount. To meet this requirement, certain roles and responsibility will be assigned, directed and executed throughout the duration of the project To foster an environment of learning through this applied research technique, certain duties will be rotated between team members throughout the term. To accomplish this, a hierarchy of responsibility is to be established.  Below is outline of the major roles needed to complete the DAARC project and accompanying key responsibilities.

Head Project Manager

• Assigns roles
• Task specific dedicated researchers
• 
  
• Develops plan for data collection to met mission requirement
• marry sensor to platform
• marry platform to mission
• Supervises flight operations
• Participates in mission briefs and debriefs

Operations Manager

• Develops all aspects of flight operations
• Preforms pre-operations checks and inspections
• Coordinates with community
• ensures all rules and regulations are being followed
• Participates in mission briefs and debriefs

Operations Technician

• Maintains and accounts for all associated ground equipment. 
• Ensures security and integrity of operational area during flights. 
• maintains flight and mission logbooks
• Participates in mission briefs and debriefs

Flight Engineer / Pilot in Command (pic)

• Maintains/ prepares flight equipment
• Conducts flight operations
• Follows laws and regulations
• Participates in mission briefs and debriefs

Payload Integration Technician

• Coordinates with project manager to assign aircraft and payload combination
• Ensures data collection capabilities matches mission needs
• Participates in mission briefs and debriefs

Product Development Supervisor/Editor

• Ensures data collection meets mission needs
• Organizes data to promote product development
• Participates in mission briefs and debriefs 

Liaison and Operational Assistant

• Provides feedback to mission coordinator for critical analysis of operations
• Acts as bridge of communication between members and course instructor
• Participates in mission briefs and debriefs 

One outcome of the applied research capstone (DAARC) is a dynamic, detailed map of the entire amphitheater. This map will be the basis for an analytical research paper for comparing flood stages of the Wabash river and its effect on the community at the amphitheater. While applying research techniques, it will be come evident what basic UAS skills are required. Some basic skills to be applied during the project include; teamwork, organization, communication, the technological understating of equipment, and application of technology correctly and efficiently towards the development of a meaningful product.

Working With Geospatial Video

Geo-Spatial Video: Understanding and Applying 

Technology for Better Situational Awareness

How valuable would it be if you could take video from a moving remote source, such as a UAS platform, and get accurate GPS location information for what that video is showing. Even more, how valuable would that information be if it was in real time or Near Real Time (NRT). A user could then monitor the NRT video streaming from an active drone and see what the drone is looking at and where it is located. This information would be invaluable to users such as law enforcement, that is actively pursing a suspect, or a wild fire control manager that is monitoring the location and direction of a forest fire and making decisions for control measures based on that Full Motion Video (FMV). Full motion video, also known as Geospatial video or “mobile mapping”, is a technology that merges GPS coordinate and location information with video (Mills, 2010). There is a difference between FMV and geospatial Video as FMV usually refers to NRT video and geospatial video has been recorded and processed.

Geospatial video is highly useful and in many regards a required part of UAS operations for firefighting, civil and military application and anywhere else video to map correlation is required. Other drone users would not benefit from the large meta data sets that FMV require. The movie industry for example, would not need to know the location as much as producing an aesthetically pleasing vantage point. For the applications where geospatial video is more useful than “normal” video, it is a dramatically improved resource. Watching video feed, especially UAS video, it is easy to get “tunnel vision” and loose all sense of direction or orientation. Its to easy for a drones camera to be zoomed in to a specific target and have the user loose all sense of position and orientation of the drone itself. An argument could be made that drone accidents have been cause by the operator loosing situational awareness due to focusing on the camera feed and not the aircraft. Having metadata that produces coordinates of what the camera is focused on and presenting that information on a digital map would provide the ability to keep the drone in LOS with the target while maintaining a safe operation.

One other example of how FMV video could be better than traditional video is that while using alternative sensors, IR, FLIR, multi-spectral etc., it may not be entirely clear what the camera is focused on. Having that information would increase accuracy as the target could be verified using GPS metadata.

Using Open-source Software to Create Geo-spatial Video

The following example of geospatial video was created using “GeoTagger Free”, an open source software application from Remote GeoSystems.  The idea was to take a video with accompanying GPS data points and the merge the two so that the location of the images could be known.  
The first step in creating a map to video product was to upload the data and video into the geo-tagging software. 

GeoTagger reference
 point selection

 Once the video and data was in the program, then the software must have a reference point to base the other data points off for the most accurate position information. Ground Control Points (GCP's) would be very beneficial for this part of the process. In absence of a formal GCP, predominate land features can be used as  GCP's for correlation of the video to the data. Once the location and video has been "linked" the software will process the video to show the location of where the image was taken on the map with a cross-hair and follow along while the video plays.  With the file loaded, the path will be colored green, after the reference point is selected the path will be orange, and when the software as process the data, the path will be color coded red ( As depicted in the following images)

Reference point selection based on video imput.. 

Software adjustments point for accuracy of points.

Final product creation with target
showing location that of video

Additional features can bee added with the purchase of the Pro version such as the metadata of the platform, altitude, heading and speed. In the free version, the only feature in the additional information window is the time.                 
The product produced is a guide that assigns data points to timestamps of the video file. It can be easily seen that this type of correlation would not be the high standards of accuracy needed for survey type of applications. This software produces a product that acts more of a guide and not exact representation of the video to the ground. Regardless, this type of information is accurate enough for many types of applications. Looking at large objects to get a general orientation like when planning or  analyzing the layout of vendors for a community event, for example.  Or searching for a moving object, this Geo spatial video could provide a location.  The video used for this demonstration can be found here: GeoTag Video.

Additional features of software

References

Mills, Jacquelin W. (2010). Geospatial video for field data collection. Applied Geography. Vol 30. Issue 4. Pg 533-547. Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S0143622810000342

Bibliography Link

Research Project towards the Integration of UAS in the NAS and MUMT Annotated bibliography.rtf

Research Project towards the Integration of UAS in the NAS and MUMT: Annotated bibliography and Research Timeline (Part 3-3)

Research Timeline (tentative)


To put an end-game to this project, a timeline must be developed. This post will attempt to attached key milestones to the project along with a final deadline for project completion. The following table depicts the general sections of this project and their respective deadlines. The biggest take away from this timeline is to understand the importance of moving forward and completing sections towards the final end goal. These are just general topics and other subsections will need to accompany each milestone. For example, the final project outline will need to incorporate what deliverables will be produced throughout this process. During each completion of milestones, this table will be updated to reflect a better more inclusive timeline.

Research Project towards the Integration of UAS in the NAS and MUMT: Annotated bibliography and Research Timeline (Part 2-3)

Annotated Bibliography

NASA Tech Briefs. (2016). Drone control: Flying the crowded skies - advanced technologies & aerospace database - proquest. NASA Tech Briefs, 40(2). https://search-proquest-com.ezproxy.lib.purdue.edu/advancedtechaerospace/docview/1766804152/9CA08D1D27434CCFPQ/12?accountid=13360


Topics covered are “see and avoid, beyond line of sight and software suites for integration of UAS”. Concepts are mentioned that plan to use sharing of information as primary means for UAS within the NAS. The idea being the utilization of a “cloud based unmanned aircraft traffic management (UTM) system”. The UTM systems has many conceptual ideas to how UAS could conduct low level operations. Concepts include digital flight plans, cellular network for information sharing of location and related metadata, and geofencing applications.

The second part of the journal mentioned human some human factor concerns. One question addressed was how could ATC personnel handle the workload and ability to de-conflict between all types of possible drones while still keeping the skies safe for manned aircraft.

Chula vista police department and cape launch new public safety drone initiative for San Diego UAS integration pilot program. (2018, Oct 25). PR Newswire Retrieved from https://search-proquest-com.ezproxy.lib.purdue.edu/docview/2124717358?accountid=13360

Identifies one of first cases of drone integration into state government law enforcement agencies for emergency response. On of 10 selected as part of IPP.

This references is short and does not contain much information. Does give way to local links for more information. (https://www.sandiego.gov/uas.)


Davies, Nicola. (2017, January 24). Up in the Air: Drone-Assisted disaster response is taking off. Fire Rescue. Firerescue.com. Retrieved from https://www.firerescuemagazine.com/articles/print/volume-12/issue-1/special-ops/up-in-the-air.html

This online journal made short points for the advantages and disadvantages for y=using drones during disaster relief. Size, efficiency and cost were all used as advantages, while collision avoidance was one of the major disadvantages. The journal cites Dr. Brent Terwilliger of the Worldwide College of Aeronautics at Embry-Riddle Aeronautical University in Florida on the problem of “sense and avoid” with regard to manned aircraft using the same area. Dr. Terwilliger continued this argument, “The major challenges to employing UAS in disaster or emergency response include de-conflicting and coordinating use with other assets in the area, including manned aircraft, and gaining regulatory approval to use UAS in a specific location.” This issues goes beyond the use of drones in disaster areas but anywhere that man and unmanned systems could potential be in the same area.


Further examples included drone selection to meet specific time and altitude requirements, training for equipment for operators and possible harsh weather endureace capabilities during disaster type conditions.


Laszlo, B., Agoston, R., & Xu, Q. (2018). Conceptual approach of measuring the professional and economic effectiveness of drone applications supporting forest fire management. Procedia Engineering, 211, 8-17. doi:10.1016/j.proeng.2017.12.132

Makes strong arguments for use of drones during fire fighting operations. defines efficiency of UAS, use with a damage to time model. Defines equations for determining economic cost during operation.

Strong arguments for difficulty in measuring economic benefit.


Cons: some content is lost in translation. Difficult to read. have to change units to familiar emperical units to understand ratio.


Manfreda, S., M. E. McCabe, P. E. Miller, R. Lucas, V. P. Madrigal, G. Mallinis, E. Dor, D. Helman, L. Estes, G. Ciraolo, J. Mullerova, F. Tauro, M. I. de Lima, Jlmp del Lima, A. Maltese, F. Frances, K. Caylor, M. Kohv, M. Perks, G. Ruiz-Perez, Z. Su, G. Vico, and B. Toth. 2018. "On the Use of Unmanned Aerial Systems for Environmental Monitoring." Remote Sensing 10 (4). doi: 10.3390/rs10040641.


Pros:Report about using UAS to improve weather monitoring techniques and practices. Highlights

GIS technologies and advancements. Compares UAS ability to gather data against traditional methods i.e. satellite images. Highlights to use of UAS to get information and images as highly beneficial to meet the needs of analysis.


General Aim of paper: "review the current state of the art in the field of UAS applications for environmental monitoring, with a particular focus on hydrological variables, such as vegetation conditions, soil properties and moisture, overland flow, and streamflow."


Makes valid points about quality of data and need to find a way to "harmonize" the data for it to reach a higher level of potential analysis.

Cons: not as relevant to integration as other reports. Speaks most about data collection technoligies and not techniques.


Targeted News Service. (2018, Sep 18). New waiver for drone operations over populated areas to help state farm respond to damage in states impacted by hurricane Florence. Targeted News Service. Washington, D.C. Retrieved from https://search-proquest-com.ezproxy.lib.purdue.edu/docview/2108801431?accountid=13360

Article highlights UAS for use during Emergencies. Focuses on insurance company ability to assess damages and “allocate resources”. Waivers for regulations that are not normally obtainable from the FAA for BVLOS and flight over people were given to the corporation (state farm insurance) to complete post emergency drone usage. Ethical concerns could be highligheted from this article; why large corporations are given waivers when other more beneficial entities are not. Company partnered with a IPP member to complete these high risk flights for private financial gain. Key points that support integration of drones are mentioned. Provides supporting points for future of safe drone operations.

Vagueness of article and lack of details of how the mission was accomplished were omitted. Lack of details provides little expect the knowledge of it was done, but not how.


Global Data Point;Amman. (2018, November 14). Simulyze conclude support of NASA's third Nationwide test of its UAS traffic Management platform, begins work on FAA's integrated pilot program. Global Data Point; Amman. SyndiGate Media Inc. Retrieved from https://search-proquest- com.ezproxy.lib.purdue.edu/advancedtechaerospace/docview/2132647134/88BD5A78326F42D7PQ/ 1?accountid=13360



Pros: Current events with regards to UTM and UAS integration. References status of 4 level of UTM.

Abstract for UTM, “NASAs Technical Capability Level tests are advancing research of prototype technologies for a national UTM system that could develop airspace integration requirements for enabling safe, efficient low-altitude operations by drones.”

Strong references for FAA/NASA IPP. Included timeline for completion of capability testing of all levels. Illustrates scope of IPP program. Has information on the company that was awarded to test for IPP

Con: Does not list all locations and project for IPP.

Valavanis, Kimon P.;Vachtseanos, George J.(2014). Unmanned Handbook: Future of Unmanned Aviation. Section: 126.1, 126.3. Retrieved from https://link.springer.com/referenceworkentry/10.1007%2F978-90-481-9707-1_95


Pros:This reference is extensive and inclusive towards Unmanned aerail vehicles as of 2014. In the chapter “Future of Unmanned Aviation” (section 126)challenges are address for the integration of UAV’s into commercial airspace alongside manned aviation. A major focus was not only the technical shortcomings against integration, but also public approval and ethical dilemmas. Most of section 126 focused on the technological shortcoming around UAS. The end of the section listed sense and avoid as major concern of UAS within NAS.

Cons: some material is dated. Information and regulations changed in 2016 that made some ideologies from this report outdated.

Yak, Shelley (2018). House transportation and infrastructure subcommittee on aviation hearing. (2018). (). Washington: Federal Information & News Dispatch, Inc. Retrieved from Research Library Retrieved from https://search-proquest-com.ezproxy.lib.purdue.edu/docview/2100337432?accountid=13360


Pros: This dictated hearing highlighted the need for UAS Integration by acknowledging the scope of UAS use and future applications. Compared number of drones registered to manned aircraft for the FAA (230,000:300,000) Next gen is mentioned as well as atc responsibilities.

Included FAA’s vision for UAS integration.

Cons:To much “fluff” and political formalities throughout document which are distracting. Long winded paragraphs that can be skipped to get to the important topics.

Yap, Basil K. (2018). Unmanned Aircraft Systems Integration Pilot Program. NCDOT. Retrieved from https://www.ncdot.gov/divWoSons/aviation/uas-integration-pilot/Pages/default.aspx


This YouTube short commercial style news update promotes the North Carolina Department of Transportation as one of the 10 selected for the FAA IPP. General direction for implementation includes package/food delivery, sensor integration and UAS in NAS integration. Provides quick details for flights beginning in fall of 2018 towards testing.

Cons: short and vague. Same as other IPP news releases. Need more information but act as a good way to gather selections on all 10 IPP locations.

Keywords

Manned, unmanned, UAS, Unmanned Aerial Systems, drones, MUMT, cooperative collection methods, integration of unmanned aviation into civilian airspace, disaster relief, wild fire spotting, NRI, sense and avoid, detect and avoid (DAA), DAIDALUS, integration pilot program (IPP).

Research Project towards the Integration of UAS in the NAS and MUMT: Annotated bibliography and Research Timeline (Part 1-3)

What is this all about.

The beginning of any project starts with the idea. The idea of this project is to provide a road map leading towards the integration of UAS with traditional manned aviation. This road map will be the framework for the utilization of unmanned systems cooperatively with other aviation and ground assets towards the safe and efficient accomplishment of high risk, and time sensitive missions. Chiefly in the realm of emergency response, fire-spotting, search and rescue etc.

The main portion of this post is dedicated to the identification and development of a research citation database. The beginning of this research, as with any other, is to build a suppository of knowledge within the field and targeted at final product. Base topics for this “annotated bibliography” include the following: Integration of UAS in the NAS, Manned unmanned Teams (MUMT), cooperative practices of data collection during time sensitive missions, crew resource management (CRM) and broad spectrum asset utilization for highly, efficient operations.

Project Considerations

Diverging away from the original capstone project proposal.

The initial project proposal was the development of a comprehensive UAS program. One that could be adapted to any prospective commercial entity delving in the idea of using unmanned aerial systems to supplement their respective business plans.  The major consideration for digressing from that project to this modified version is the desire to create a research project and not begin a commercial endeavor. I plan on the future to promote myself as an advisor of UAS and believe that by creating an open project now would compromise any future marketability as a UAS advisor.

I believe there is a plethora of unexplored material for the use of sUAS during emergency responses. Emergencies have unique components with respect to aviation operations and assets. The deployment of such assets relies heavily on the efficiency of data collection, quick reaction time, and overall safety of deployment, that is just not present in any other form of drone application. The idea that lives could potentially be saved by their use presents a draw that is unsurpassed by any other occupational application. Furthermore, it is frustrating to know that current technology is capable, but regulations have not caught up enough to allow this type of integration. It will be the focus of this research to add to the integration of UAS into the NAS, as well as develop a methodology for cooperative asset utilization for manned/unmanned missions.

Why APA.

For this bibliography and subsequent research paper, the citation style most accurate is the American Psychological Association 6^th additions (APA). APA supports the social sciences, focuses on date as a way to track currency and relevance to the topic and is generally accepted as the standard citation throughout Purdue University. The target audience for this project is for those conducting research on the topics and review by academics within the field.