Web Photogrammetry – experience and practical examples.

Zygmunt Paszotta

Department of Photogrammetry and Remote Sensing, Faculty of Geodesy and Land Management, University of Warmia and Mazury, Oczapowskiego 1, 10-719 Olsztyn, Poland, paszotta@uwm.edu.pl

Abstract: Interactive network photogrammetric applications have appeared along with the development of the Internet. An example of an application which can view Earth images and execute measurements of distance in these images is Google Earth. The author claims that it is possible and important to create more advanced photogrammetric network applications. In this paper, the structures (architectures) of selected photogrammetric network applications are presented. In these kinds of applications, internet browsers and Virtual Machine JAVA (VMJ) are applied by users in what are known as web applications. The advantages and limitations of such solutions are also described. The author presents examples of the determination of elements of aerial photo orientation, measurement executions, DTM and an orthophotomap generation via the Internet. It is possible to access these examples from the web site of Work Group IV/5 ISPRS

Keywords: Internet/Web, Photogrammetry, Application, Programming, Web-Based, Learning, Teaching

1. Introduction

One of positive features of the Internet is the possibility of disseminating different types of Earth images. It is a source of popularity for a portal such as Google Earth, Google Maps, Windows Live Search or NASA World Wind. Usually, users neither have suitable photogrametric photos nor appropriate software to process them. In particular, this concerns land surveyors (geodetic), architects, town planners and students, who want to study the basics of photogrammetry. There are internet solutions which can execute photogrammetric tasks interactively by using an internet browser. They are described as Web Photogrammetry. The ARPENTEUR System is an example of a Web application in which measurements in photogrametric photos belonging to users of this system are possible (Drap, Grussenmeyer 2004). It is aimed mainly for terrain photogrammetric purposes, what is strictly connected with limitations of the storage size of digital images sent to the system. The current data transfer rate is responsible for the fact that import or export (sending, transfer) of the whole aerial photo to or from a server takes a lot of time. Thus, solutions applied, for example, in Google Earth rely on gathering and storing images on the server. Only their fragments are sent to users. It guarantees the attractiveness and high speed of such portals. It should be emphasized that Google Earth is not strictly a web application but a network application. Other example of a Web application created by Google is Google Maps. The author tries to explain the difference between these terms in a further part of the article.

In constructing a Web photogrammetric application, it can be assumed that photos will be stored on different computers – data servers. Software which is necessary to read data from these servers will be installed on another computer called an application server. Moreover, users will be communicating with the application server by means of their own software. The above description presents a distributed application scheme. In view of safety and ease of use, it is assumed that a user is only using Web browser. This solution has enormous didactic merits. It can be supposed that technology based on distributed applications is one of the scientific development paths of image processing methods, along with aerial photos and satellite images. Therefore, it is an object of interest of the ISPRS Commission IV.

2. Architecture of photogrammetric internet applications

The specificity of Web Photogrammetry arises from the fact that all tasks are performed in the computer network. Computers and other hardware such as plotters or printers are connected to exchange data. The photogrammetric data can be stored on expensive, well-equipped servers, but the majority of the program logic and user interface can be done by a client (a program accesses a remote service on another computer by network) on the user PC machine. In general case a client as a program does not have to be executed in the browser’s environment. It can be previously downloaded and installed on computers. Thus, it is called a network application. The concept of Web application includes software that is not installed on a computer but the package is available as an applet by a Web browser.

During construction of photogrametric applications client-server technology and programming in the Java language is particularly useful. The classic scheme of the structure of an application in client-server technology is presented in figure 1.

Figure 1. Architecture of Web photogrammetric application.

We assume that a system customer is only using a web browser. It means that when a web page is request the server can send not only the HTML code of the web page but also small programs – called applets. In principle, a server provides data and a client receives it. However, the assertion that a client initiates communication by making a request and a server performs operations to respond appropriately to it is more correct (Harold 2004). Actions on the server side are conducted by programs called servlets. Applets and servlets are written in the JAVA programming language and compiled to bytecode, which at runtime is either interpreted or compiled to native machine code for execution. In order for photogrammetric applications to act correctly it is necessary to have a WWW page server, e.g. Apache, and an application server, e.g. Tomcat, to execute servlets. Additionally, users have to install Java Virtual Machine to execute applets on the user’s side.

3. Aspects of network programming for photogrammetric purposes.

Digital photogrammetric workstations are usually equipped with advanced technology. The process of creating internet photogrametric applications is not based on copying or translating the source codes of such systems. It is necessary to build new programs to conduct definite tasks in an overt manner. They ought to fill gaps in of the user’s existing software and take into consideration the properties of the network structure (Harold 2004). All activities connected with programming such applications depend on the type of software on the client side. In the case of Google Earth it is a network application. Although ARPENTEUR runs on a Web browser the users must install some plug-ins. Downloading and permanent installation of software elements requires confidence in the software creators. Organizational and psychological aspects are significant here. In the current case, it is a thin client and it works in browser environment without additional interference to software (programming) or the operating system on a user’s computer. Thus, the creation of such applications requires writing applets and servlets with suitable communication between them (Hunter 2002). It is essential to understand how the HTTP protocol works because it is used in sending requests and answers (Hall 2003). Generally, all forms of communication allows the transfer data of different size and types such as images, elements of photo orientations, measured coordinates and calculation results. Applets should also enable simple service of the application by means of a friendly user interface. On programmer’s side, it means the necessity of using suitable AWT or SWING event service software (Horstmann 2005). Servlets, as software installed on a server have the task of servicing all users. However, it is essential that servlets and applets, through code written in JAVA, must correctly perform certain photogrammetric tasks. Diagrams presented in a further part of the article illustrate this.

The fundamental problem of Web Photogrammetry is also the necessity of transferring whole photographs, which usually require more than 300 megabytes of storage size. Not only is it a technical aspect, but it also involves methods of programming. In the previously mentioned system – ARPENTEUR - user’s files (photographs, coordinates of control point, etc.) must be uploaded on the APENTEUR server. But the image size is limited to a few Mb (Drap, Grussenmeyer 2004). Thus, solutions are needed that ensure to deliver a desirable fragment of photographs via the Internet and visualize it on an appropriate scale on the client side within a few seconds after the fragment was established. In our examples of a Web application servlet extracts and delivers selected fragments of images.

Summing up, the challenges facing programmers are the following:

- lack of possibility to use disk storage

- necessity of transmission and processing of fragments of images/photographs

- low execution speed of programs in JAVA

- low transfer rate between clients and servers

- necessity of minimization of applet code

The price which is paid for overcoming these difficulties is the limitation of the functions of internet photogrametric applications. On the other hand users gain benefits such as:

- no necessity to install and update software

- reliability and safety of application resulting from running/executing it in the internet browser’s environment

- small/minimal system equipment requirements

- possibility of using large data resources

- high didactic/educational merits/values

- capability of wide and interactive publication of the results of photogrametric projects.

These advantages to a wide extent satisfy scientific and didactic needs (Pateraki, Heipke2005). Nowadays, it is difficult to demand in order to applications can execute photogrammetric tasks on the photos belonging to users. Aside from data transfer rates, users have many different reasons not to send their photographs to other computers (servers). It is simply illegal in some countries.

The idea of wide dissemination of photogrammetric knowledge over Internet is good in theory, but is not currently commercially profitable. It can be realized by means of various types of applications. At present, there are only a few interactive applications, which allow to execute real photogrammetric tasks. More common is the creation of thematic tutorials or on-line courses. Unfortunately, they usually have limited interactivity (Pateraki 2005). Such solutions are easier and cheaper to produce. In a further part of this publication examples of web photogrametric applications that may be used for e-learning of Photogrammetry are presented.

4. Internet Photogrammetry based on web photogrammetric application dedicated to aerialphotogrammetry.

The several elementary photogrammetric tasks, which have been made accessible on stand-alone digital workstations to date, have been programmed based on the architecture of web application presented previously. The author of this publication has created a set of web applications for aerial photogrammetry. They are installed on the server of the Department of Photogrammetry and Remote Sensing of the University of Warmia and Mazury (www.kfit.uwm.edu.pl/zp). Using a browser, the following tasks can be performed:

- choose and visualize the stereopair of a block of photographs,

- visualize fragments of othophotomaps and take measurements on them,

- determine elements of photograph orientations (Paszotta 2003),

- visualize a 3D terrain model by means of an anaglyph,

- measure the pixel coordinates of homologous points, map them automatically and determine their coordinates in a terrain coordinate system,

- automatically measure terrain coordinates of set of points for generation or verification of a Digital Terrain Model (Paszotta, Szumiło 2006),

- interactive generate orthophotomaps over the Internet.

4.1. Transformation from a pixel coordinate system to a fiducial coordinate system.

In order to slightlyshorten the activity diagram it was assumed that the user chooses a photograph stored on the server and the coordinates of fiducial marks are sent him. However, the measurement of the pixel coordinates of fiducial marks is the most essential problem. This process becomes largely automated (fig. 3). All stages of the computational process are shown in figure 4, and the results of the example computation are given in figure 5.

Figure 3. Automatic measurement of the coordinates of fiducial marks.

Figure 4. Activity diagram shows steps in the process of determination of the transformation from a pixel coordinate system to a fiducial coordinate system.

Figure 5. Results of measurements and computation of inner orientation parameters.

4.2 Relative orientation of stereopairs and generation of anaglyphs.

A crucial solution allowing for the Internet realization of this problem is the possibility of using the image pyramids over the Internet. We assumed that such pyramids exist on the server and a pair of photographs was chosen. A sequel to the process of executing the relative orientation of photographs and the anaglyph construction is described with the aid of a diagram in figure 7.

Figure 6. The identification of homologous points in the process of relative orientation of a stereopair.

Figure 7. Activity diagram in the process of relative orientation of a stereopair.

During the process of measurements, coordinates of at least six homologous points are collected in the table (fig. 6). By correctly arranging and carefully

measuring homologous points enough accurate parameters of relative orientation can be fixed in the third or fourth iteration.

Figure 8. Elements of relative orientation in the following iterations.

An anaglyph can be also created over the Internet after running the application and performing the activities described in figure 7. Users do not have to take measurements of homologous points and calculate elements of relative orientation each time they visit the Web site. The set of test data is stored on the server. If users do not execute the relative orientation of a stereopair the epipolar images are constructed based on elements determined previously.

4.3 Determination of terrain coordinates.

In order to carry the task of determination of terrain coordinates over the Internet, it is necessary to have an opportunity to use fragments of photographs from the different levels of an image pyramid. . The organization and exchange of data between the client and the server should be able to measure the pixel coordinates of homologous points and transform them to the next coordinate system using elements of image orientation. It was assumed that relative and absolute orientation or exterior orientation of a stereopair had been executed earlier. The parameters of those orientations and coefficients of affine transformation are stored on the server. If, as a result of applying the image matching method, there is the large set of homologous points, their terrain coordinates can be used for construction of a Digital Terrain Model. This problem has already been an object of research and was published in detail (Paszotta, Szumilo 2006). In this article a different approach of photogrametric determination of terrain coordinates of points is presented. Users indicate vertexes of a polygon on the first photograph and approximate location of these points on the second one. After these actions, the vertexes of the polygon are matched. Terrain coordinates and the area of a polygon are calculated automatically. Figure 9 presents the stages of the task in the form of an activity diagram. The results of its implementation are shown in figure 10.

Figure 9. The solution of interactive squaring of a polygon.

Figure 10. Implementation of the squaring of a polygon.

4.4 DTM and an orthophotomap generation.

The process of creating orthophotomaps from aerial photographs is complex and requiring the specialist software on the digital photogrammetric station. However, it turns out that this process with certain limitations can be executed by the Internet. The algorithm of orthoimage generation is presented with the aid of UML diagram (fig 11). The terrain coordinates of points, which are being used to create digital terrain model (DTM), are measured and calculated in the automatic way. However, the area of the orthophotomap is small and limited by dimension of the window of the Internet browser. Thus, in the process of orthoimage creation via the Internet the author assumed a simplified DTM in the form of the plane. Orthorectification is made on the server side but the grid coordinate system is superimposed on the orthophotomap by means of applet on the client side. Besause of resampling, the created orthoimage has a worse quality than a source image. Therefore, the source photograph with the system of coordinates is also presented. In both cases the image and terrain coordinates of point shown by the cursor are calculated and printed in the header of the Internet browser window.

Figure 11. Activity diagram in the process of DTM and orthophotomap generation.

Figure 12. Visualization of set of points for approximating plane and determination of the plane equation.

Figure 13. Visualization of the orthophotomap in the internet browser window.

5. Conclusion.

The development of Web Photogrammetry methods is connected with Internet evolution and technical progress in digital photogrammetry. Web Photogrametry has enormous didactic and scientific values and is an effective tool of photogrammetry popularization. The slowness of implementation of web photogrammetric applications stems from the lack of direct commercial success of such activities. Nevertheless, there are a lot of possibilities to use these solutions for educational purposes as well as in Integrated Administration and Control System (IACS). The significance of web photogrammetry creation was also perceived by International Society of Photogrammetry and Remote Sensing. This organization created a new working group (Web-based Geoinformation Services and Applications) as a part of IV Commission. The described applications located on the Department of Photogrammetry and Remote Sensing server .They are also accessible from the ISPRS WG IV/5 too ( http://www.commission4.isprs.org/wg5/).

6. References:

Drap,P., Grussenmeyer,P., Gaillard,G., 2004, Presentation of the web based ARPENTEUR tools: towards a Photogrammetry based Heritage Information System, The International Archives of Photogrammetry and Remote Sensing, Istanbul, Turkey Vol. XXXV-B6, pp.123.

Hall M., Brown L., 2003, Core Servlets and JavaServer Pages.vol. 1: Core Technologies. PrenticeHall PTR, 2 edition, pp.130-132, ISBN-10 0130092290

Harold E.R., 2004, Java^TM Network Programming, 3^rd Edition. O’Reilly Media Inc., ISBN 978-0596007205

Heipke Ch., 2005: Web-Based Photogrammetric Image and Geospatial Services – an Overview Photogrammetric Week 05, Wichmann Verlag, Heidelberg p 157-164

Horstmann C. S., Cornell G., 2005. Core Java Volume II Advanced Features. 7^th Edition, Prentice Hall PTR, ISBN-10 0-13-111826-9

Hunter J., Crawford W., 2002. Java Servlet Programming. 2^nd Edition O’Reilly Media Inc. ISBN 978-0596000400

Paszotta Z. 2003: Exterior orientation and other photogrammetric solutions through the Internet. Automatic Georeferencing of Aerial Images by Means of Topographic Database Information. Aalborg University, pp. 25-32. ISBN 87-90893-47-6, ISSN: 1397-3169

Paszotta Z., Szumilo M., 2006, Application of a statistical test of hypothesis to check DTM accuracy over the Internet. EuroSDR Official Publication No 51.