Linear Algebra: small survey of books

It is sometimes useful to repeat some fundamentals of mathematics, especially when it has not been used for a long time. For the study of coherence estimation in complex-valued data I had to review and refresh my Linear Algebra skills. For that I needed to study several topics and the result of it will be this small survey of books, handbooks and articles that helped me in such challenging study.

General-purpose books

First of all, one need a good general-purpose books for initial refresh of Linear Algebra. I think that Gilbert Strang's book [1] is among the best for systematic study. Although sometimes (especially in the appendices) book became too verbose (lots of exotic examples) or too brief (the reader must guess of how to actually calculated many types of decompositions), Strang's book is a good starting point anyway. It covers all basic topics except quadratic forms; very short introduction to SVD and Pseudo-Inverse. It is noticeable that Strang's video lectures are sometimes way better than his own book.

The book ``Linear Algebra'' written by J. Hefferon [2] is great for start and initial understanding of Linear Algebra. Clearly written and with lost of examples, it is a must-read book to quickly remember some base material of Linear Algebra. Moreover, there are some applications after each chapter. The textbook [2] is useful as a teaching material because a lot of examples and deductive approach of material's explaining. Examples in this book are very insightful (least-squares, crystals, economic examples and so on). Coverage stretches from basic to eigenvalue decomposition that is not enough (no SVD, Pseudo-Inverse and Quadratic forms at all).

Specialised books

One of the most necessary books for me is Matrix Algebra from a statistical perspective [3] written by D. Harville. The book contains most of necessary topics for Linear Algebra applications from the statistical and engineering point of view. For instance, the book contains insightful chapters about matrix differentiating (Chapter 14) that is very helpful.

The book written by Hoffman and Kunze [4] is a good starting point of studying bilinear and quadratic forms. The book of Marcus and Ming [5] is a good reference for matrix inequalities and other special topics of algebra. A very good book [6] written by Gene H. Golub contains interesting topics and discussion of Schur decomposition and Pseudoinverse.

For several advanced topics and pure mathematical proofs, I want to mention Radjendra Bhatia's Matrix Analysis [7] book. The questions such as Spectral variations of Normal matrices and Majorization were helpful for me.

Articles and notes

Many properties of product and sum of pseudoinverse matrix are described in articles. One particular article that discuss the properties of product of pseudoinverse that is (AB)^+ =A^+B^+ is Taussky's article [8].

A very good introduction to differentiating of matrices is written in small ``Notes on Matrix Calculus'' [9] by Paul L. Fackler from North Carolina State University. That is one os the most easy, clear and bright introduction to matrix differentiating that I could found.

The small book about Toeplitz and Circulant matrices with very good introduction is written by [10] (can be download from the Internet). This book allows to use the necessary properties of Toeplitz matrices in applied science without digging in pure mathematical folios. Writing style is clear and shiny with reasonable amount of examples in statistics and signal processing.

And of course, one should definitely read Schur's original paper (available in digital form) about Schur decomposition [11].

Reference books and Handbooks

A truly great reference is Leslie Hogben's handbook [12] that contains the most of material of Linear Algebra. Very concentrated material, with numerous links to other books and articles, ``Handbook of Linear Algebra'' is indispensable on reference and quick recalling some additional properties and relations of Algebra's objects.

Although the book by Horn& Johnson [13] ``Matrix Analysis'' appears in practically any reference sources, it is not an easy reading material. One should not read it from front to back, but rather selected topics. The material is well organised but is very dense: Horn&Johnson book is rather handbook than a textbook.

Useful and helpful handbook that contains many inequalities and interesting properties of Linear Algebra's objects is The Matrix Cookbook [14] written by K. B. Petersen and M. S. Pedersen. Although there are mentions the the Cookbook contains many mistakes and inaccuracies, it is useful and may be utilised as quick reference. For instance, it contains short but bright description of the matrix differentiating.

Bibliography

1

Gilber Strang.
Linear Algebra and its Applications.
Thomson Learning, 1988, 3^d Edition.

2

Jim Hefferon.
Linear Algebra.
2000.

3

D.A. Harville.
Matrix algebra from a statistician's perspective.
Springer Verlag, 2008.

4

K. Hoffman and R. Kunze.
Linear Algebra.
Prentice-Hall, Englewood Cliffs, NJ, 1971.

5

M. Marcus and H. Minc.
A survey of matrix theory and matrix inequalities.
Allyn and Bacon, Boston, 1964.

6

G.H. Golub and C.F. Van Loan.
Matrix computations, 1996.

7

R. Bhatia.
Matrix analysis.
Springer Verlag, 1997.

8

O. Taussky.
Commutativity in finite matrices.
American Mathematical Monthly, 64(4):229-235, 1957.

9

Paul L. Fackler.
Notes on matrix calculus.
North Carolina State University, 2005.

10

R.M. Gray.
Toeplitz and circulant matrices: A review.
2006.

11

I. Schur.
On the characteristic roots of a linear substitution with an application to the theory of integral equations.
Math. Ann, 66:488-510, 1909.

12

L. Hogben.
Handbook of linear algebra.
CRC Press, 2007.

13

Roger A. Horn and Charles R. Johnson.
Matrix Analysis.
Cambridge University Press, 1985.

14

K.B. Petersen and M.S. Pedersen.
The Matrix Cookbook.
Technical University of Denmark, 2008.
20081110.

Newcastle University network: using under Linux

After another update my WinXP on the Dell notebook started to load so ridiculously slow (~45 minutes) that I cannot sustain it any more. All of that bells and whistles, such as anti-virus, anti-spyware and so on do whatever they want and do not allow me to work.



It is funny but IT service seems to close all holes so you are unable to install Linux or FreeBSD on Dell's laptop that goes from the University. But we can use our own laptop - we only need to know Dell's MAC address.

Changing MAC address in Debian
That's pretty easy: just type as root

# ifconfig eth0 hw ether xx:xx:xx:xx:xx

or correct the /etc/network/interfaces file like:

auto eth1
iface eth1 inet dhcp
hwaddress ether 00:00:00:00:00:00

And now make /etc/init.d/networking restart
That's it!

Using your Browser to surf web
For that we should tell Firefox (or other browser that you prefer) to use University's proxy server that is proxy.newcastle.edu.au:8080

For that use in Firefox: menu Edit -> Preferences -> Advanced -> Automatic proxy configuration and then type there:

http://proxy.newcastle.edu.au/proxy.pac

For now on you can use your browser to surf web from Linux.

Printing from Linux to UNI's printers [TBD]

How to use NUmail via email client

Yesterday's technologies - tomorrow!

Re-inventing the wheels since 1982!

// the slogan of Microsoft (c) Corporation (TM)

Abstract

A new email system of the University of Newcastle (NUmail) is proven to suffer from that-is-to-say web-interface from Microsoft (c) Corporation (TM). As an alternative way of working with email without dealing with web-interface, email clients such as Microsoft Outlook or Mozilla Thunderbird may be considered. It will be shown in this paper that using IMAP protocol it is possible to synchronize the content of the mailbox with server mailbox. Thus one can use benefits from email client (normal human interface) and from web-interface (being away from workplace).



In this paper, a way of send/receive email from NUmail account without web-interface is proposed. Configuration options for IMAP/POP3 (incoming mail) and SMTP (outgoing mail) for NUmail are provided. Experimental results on successful configuration OpenSource email client Mozilla Thunderbird are presented.

Introduction:

How to configure mail-client for sending and receiving NUmail?

First of all, this note is concerned on Mozilla Thunderbird v3 (main window is shown in Fig.1), but with little imagination one can easily configure any other mail client. Be sure that your mail client is able to change connection's security method - otherwise you can't send or receive messages. Thunderbird can do it; besides it is OpenSource, License-clean, free and cross-platform (Mac and Linux users are welcome) - why no to try it?

Figure 1. Main window of Mozilla Thunderbird email client -

here our mail will be mirrored.











Experimental results

Yes, it took several hours from Rodrigo and me to make it works. I'll try to describe the process as simple as possible - actually, it requires only a few pushed buttons and mouse clicks.



No matter of which mail client do you use, first of all let's log in MSLive mail under your account (we need it to obtain correct mail server's parameters). In the top-right side of the screen you can see round blue HELP button like that: (?) <-- near this button there is a small down-arrow (See Fig. 2).

Figure 2. Actual parameters of email server's options are here.



Push that small down-arrow, and that in the opened menu - About. That will open a window with supplementary information about current letter like that:



Authentication type associated with this Outlook web application session:

Public login: No External



POP setting:

Server name: pod51004.outlook.com

Port: 995

Encryption method: SSL



External IMAP setting:

Server name: pod51004.outlook.com

Port: 993

Encryption method: SSL



External SMTP setting:

Server name: pod51004.outlook.com

Port: 587

Encryption method: TLS



That's what we need! Now we should use those parameters (especially selected with bold font) for our email client.





Experimental results: Receiving mail from UON via mail client

The parameters for incoming mail are as follows (from the window as described above):

Mail protocol: IMAP

Incoming mail server (IMAP-Server): pod51004.outlook.com

Port: 993

Connection Security (for Thunderbird): SSL/TLS

Secure authentification: NO

login: c1111111@uon.edu.au <--- replace 111111 with your StudentID number

password: accesskey+birthdate <------ your usual password for access to UNI webservices

Note: you can use POP protocol as well, but IMAP has very nice features like synchronizing folder's content. This means that in your mail client you will have a duplicate of server's mailbox content (e.g., you delete all messages in trash bin and same occurs on the server).



The actual configuration for my Thunderbird mail client looks like that:

Figure 3. Thunderbird's configuration for incoming mail (in order to receive mail without web-service).



If all steps and parameters performed correctly, you can receive all of your mail from the NUmail account. But we want to send our mail either, so we need to configure outgoing mail settings, too. That's not difficult and actually is almost similar to the configuration if incoming mail. Steps for configurations are provided below.





Experimental results: Sending mail from NUmail account

Now we have to configure the parameters for outgoing mail. The parameters for sending mail are the following:

Mail protocol: SMTP

Incoming mail server (IMAP-Server): pod51004.outlook.com

Port: 587

Connection Security (for Thunderbird): STARTTLS

Secure authentification: NO

login: c1111111@uon.edu.au <-- replace 111111 with your StudentID number

password: accesskey+birthdate <----- your usual password for access to UNI webservices

Yes, the connection security method for sending mail is different: in Thunderbird, it is STARTTLS. After that you can send our mail.



Figure 4. Thunderbird's configuration for outgoing mail (in order to send mail without web-interface).





As you see, if all configurations are correct, we can send and receive email from the account by email client and avoid to log in to that web interface.





Conclusions and Discussion

That's not a rocket science, right? But wait a minute - there is one more feature! Do you remember that stupid restriction of sending UON mail only as C1111111@uon.edu.au address?



With email clients you can override it: just write down in your mail client's account settings normal addresses like ali.baba@uon.edu.au - and for now on you can send your mail with your normal address.



Anyway, your address like name.surname@uon.edu.au is only an alias to c1111111@uon.edu.au so you can receive email on both addresses - and there is no excuse of stupid limitation of sending only from c111111@uon.edu.au mailboxes.







Acknowledgments



I would like to thank my colleague Rodrigo for his heroic efforts, titanic fortitude and detailed investigations of mail server's parameters (Rodrigo, ask me for a beer at any time). Without his precious information we all (including the author of this text) would still wasting our time fighting with that web-interface.



This work was performed under financial support of THE UNIVERSITY OF NEWCASTLE INTERNATIONAL POSTGRADUATE RESEARCH SCHOLARSHIP (UNIPRS) that is infinitely and truly appreciated by the author.



Any suggestions and corrections are welcomed in comments below (no registration needed).

Google Scholar and import into BibTeX

As many of us already know, Google has special search engine for search of scientific publications - Google Scholar. As a specialised version of Google search engine, Scholar provides many useful features:

1. you can search the specified articles and exclude patents;
2. search within defined dates is possible;
3. search only articles in the following subject areas;

All of these features are in Advanced Scholar Search link and it is quite handy.

But the real treasure is hidden in Scholar Preference page: at the bottom of the page you find Bibliography Manager and (that is the pearl) Show links to import citations into option.

That's it! Select it and start searching - any book or article. For instance, let's find Trigonometric series Handbook. Now look carefully at the bottom of the results:

Cited by 3597 - Related articles - All 6 versions - Import into BibTeX

That's right - we can instantly get a BiBTeX file and paste it in our favourite BiBTeX manager. It saves a great deal of time for me, so I hope that it would be useful for you, too.

Nip2 - the advanced images analysis tool

The nip2 is a unique images analysis tool - it is not a conventional graphical editor like Adobe Photoshop or The GIMP.

The nip2 approach: each processing result is a cell
Nip2 has a non-trivial yet productive interface that is a kind of mix between a spreadsheet and a graphical editor (imagine a cocktail with Photoshop and Excel). The result of any operation is put down into a cell and you can make references to any cell (that is an image after some processing operation). For example, you can select the area of interest (cell A2) on the original image (cell A1) and apply some filter to region of interest that will be A3 cell. Such an interface allows you to quickly recalculate the resulting image if any parameters of previous filters have been changed.
The screenshot illustrates such paradigm.


Hence the nip2 is not a conventional graphical editor but rather a graphical analyser. Nip2 supports the most useful graphical formats such as TIFF, JPEG, PNG, PPM, as well as scientific formats like MAT (MATLAB's matrices) and convolution matrices. Thanks for using VIPS library, the nip2 allows to view extremely large images very fast: you definitely appreciate such feature is you process scientific data.

What is expected to find in NIP2 and what is not
To reiterate, the nip2 is rather a scientific images analysis laboratory than just another raster images editor. So there are no tools for layers and masking (and such tools are not needed). But when one deals with large images such as panorama, nip2 is priceless. Moreover, there are many advanced images processing algorithms that are hardly to be found in conventional editors: morphological images analysis, Fourier transform, statistical tools, and many other.

Few words about the interface of nip2
As above, the spreadsheet-like paradigm of nip2's interface allows you to change filter's parameters and quickly recalculate the resulting image. I'm going to show this on an example.
For instance, there is a sequence of filters that you need to apply on image - changing one parameter results on the destination image. Then you need to apply the filtering sequence for another image - and here the nip2 to rescue you: just right-click on any cell in nip2 and select "Replace from file".
That is enough for recalculation of the whole filtering sequence.


Intensity-scale change for images viewing
This is a very helpful feature when you need to view an image with specific grey levels (e.g., 12-bit raw images from the digital camera).

In order to change the intensity scale of the image, just left-drag to set number of brightness or contrast magnification. It does not affect on the real image's values but just for viewing.


Quick zooming
If you need to zoom-in or out quickly, just hold CTRL button on the keyboard and turn mouse's wheel.


Hotkeys in в nip2
The assignment of the hotkey for any menu function is really easy: just open nip2's menu, select the item and (when the item is selected) press a hotkey combination. The hotkey combination for that function will be assigned instantly.


Fast region selection
If you need to select the region of interest on the image, just hold CTRL keyboard button and start selecting the area on the image. A new cell that contains the selected appears instantly in the current column and the next number (e.g., if the current column is B and the last number of cell is 14, cell B15 with the region appears).


Quick scrolling of the image
Use the wheel, Luke! When viewing the image in the window, one can scroll the image up-down using mouse wheel. Moreover, holding SHIFT button and scrolling moves viewing area left and right, so you can completely neglect scrollbars.


Images analysis in nip2
Using nip2, one can perform advanced images processing and analysis by such tools as Fourier transform, correlation analysis, filtering and morphological analysis. The most common examples that I use daily have been provided below.

Fourier analysis in nip2
In many cases it is necessary to look not only on the image but on its Fourier spectrum. In contrast with conventional images editors, there are no problems in nip2: just use Toolkits - Math - Fourier - Forward and enjoy. You should nonetheless take into account that for large images Fourier transform can take a long time (~10-15 seconds depends on CPU's horsepower).

The reverse Fourier can be performed likewise using Toolkits - Math - Fourier - Reverse .


Images' histograms in nip2
A histogram can deliver the information about how much pixels of the same level contains the image being analysed. Indeed, this is a very useful feature, and you can find it in Toolkits - Histogram - Find - One Dimension.

As a result, we have a beautiful and informative histogram for the image.


Images edition in nip2
Of course you can edit and transform images in nip2 but some of the functions can look like kind a philosophical.

Cropping
There are two ways to crop the image: use Toolkits - Image - Crop or just open the cell with the image and select the region of interest.

Or, using menu one can select the region of interest likewise: File - New - New Region.


After that, you can save the cropped image by right-click and select "Save image as".


Threshold
The threshold function is concealed in the menu in Toolkits - Image - Select - Threshold.

Joining images in nip2
One of the most exciting feature of nip2 is joining images. While Photoshopers and Gimpers are buying heaps of RAM for their computers, nip2 users can join large images easily. Just use Toolkits - Image - Join - Left to Right or Toolkits - Image - Join - Top to Bottom and what we have:

This is much more easy and way faster than in Photoshop or Gimp: I have glued together 10x10 images each of which is 3000x2000 pixels on a notebook computer with only 512Mb RAM.


Tilt brightness
It happens sometime, and brightness tilt is very boring effect (i.e., when you analysing images from microscope, it is difficult to capture it without such brightness artefacts). But using nip2, one can correct tilt brightness using Tools - Filters - Tilt brightness.
Such function restores correct illumination on the image (to some extent, of course).

Conclusion
This post is actually a collection of my favorite tips and tricks of work in nip2. I'm going to update this post from time to time. And, of course, I would like to thank John Cupitt, Kirk Martinez, and Joe Padfield for such a great program!

Optical discs with large information capacity... and where they actually are!?

I have received recently OPN Optics & Photonics News journal and read a small reference to recent advance in large-capacity discs. In particular, Australian researchers at Swinburne University of Technology (Hawthorn, Victoria, Australia) have proposed a new type of optical disc that can store 1.6 terabytes of data.

Developed by professor Min Gu and colleagues, the technology uses the unique properties of surface plasmons in gold nanorods to take advantage of information in five dimensions: the three spatial domains, wavelength and polariza­tion. The nanorods, which are coated in polyvinyl alcohol and mounted on a glass substrate, can be selectively recorded in layers by laser light, due to their unique optical and photothermal properties. Min Gu and his team have recorded ten layers and believe up to 100 may be feasible, for a potential disk capacity of 7.2 ТВ.

We all hope that such discs can be used not only by military/financial institutions but for average users, too. Unfortunately, large-capacity discs are tend to demand highly precise equipment, or very special materials, or something else that makes a mass-production difficult or even impossible. Again, such researches are very promising, but for most of these projects are not commercially successful. For instance, many of us remember the Holographic Versatile Disc by Optware. There were many promising ideas, hopes and dreams, but in the end - the only one wiki's page...


Fig.1 Optware's Holographic Versatile Disc™ (HVD™) disc structure.

Fig.2 Read / Write system


Fig.3 Holographic Versatile Disc™ (HVD™) on which digital movies were recorded (left). The disc diameter of 12 centimeters is equivalent to those of CD and DVD.

EMVA1288 Standard

The European Machine Vision Association Standard 1288 is designed for unification of description digital photocameras and photosensors' characteristics. This is an open standard that provides a framework for estimation of camera's characteristics. The EMVA1288 standard is build modularly and contains all necessary recommendations and comprehensive mathematical formulation of how to measure and estimate photosensor's characteristics.

Every module of EMVA1288 Standard consists of mathematical model, the experimental setup, calculation steps and recommendations of how to publish the results of measuring. Currently (version 2.01A) there are two modules for the EMVA1288 standard: Module 1 "Characterizing the Image Quality and Sensitivity" and Module 2 "Linearity and Linearity Error".

In the Module 1 "Characterizing the Image Quality and Sensitivity of Machine Vision Cameras and Sensors", the procedure of how to characterize the temporal and spatial noise of a camera and it's sensitivity to light is described.

In the Module 2 "Linearity and Linearity Error" is described the method of estimation of area and linescan sensors/cameras for which the output signal is expected to be directly proportional to the impinging photon flux (exposure). Although this module is optional, it may be useful for estimation of the real dynamic range of the photo sensor.

The EMVA1288 Standard was re-typesetted in LaTeX format as the more appropriate format for scientific use. The latest LaTeX version of the EMVA1288 Standard can be downloaded from these mirrors:

• mirror1
• mirror2

As for concluding remark, I can additionally say that EMVA1288 Standard is useful not only for machine-vision cameras but for consumer-grade cameras, too. RAW data from the consumer-grade cameras, after appropriate conversion by such software as dcraw, can be used for characterisation of consumer-grade camera as a measuring device.