Library & Technology Services of Lehigh University maintain some brief instructions and an FAQ for the University's supercomputer, Vega. This is intended as a supplement to those materials for people who need a more complete explanation of how to use these resources, or how to use them more efficiently. This document is in no way associated with LTS or any other University department, and I am responsible for all content.
Vega and other High-Performance Computing resources are designed and purchased or leased for rather specific purposes. A PC from, for example, a public computing site or the library, has many user-intensive functions: reading e-mail, word processing, and web browsing to name a few. Vega is capable of performing each of these tasks as well, but this is not its purpose. Rather, it is to be used for scientific computing. What this generally means is that vega will be able to perform long-running and complex calculations that would not be possible for a typical PC.
People may have a misconception that they will be able to do user-intensive tasks more quickly if they use a supercomputer instead of a public PC. This stems from a false belief that a supercomputer should be much faster than a personal computer. In fact, most tasks that a typical computer user executes would finish more quickly on a recently-purchased PC than on vega. The advantage of a supercomputer is that it may work on many jobs simultaneously and can fill unusual resource requirements.
In the early days of scientific computing, when resources were especially expensive, users submitted their jobs to an administrator, who decided what would be processed and in what order. Now that computing power has become significantly less expensive that human resources, users are expected to make responsible decisions themselves. When deciding whether or not it is appropriate to run a job on vega, please consider the following questions:
For smaller jobs or tasks that are not appropriate for the supercomputer but require a UNIX-like environment, the University also maintains a number of Linux-based workstations, which are less likely to be in use at any given time.
Processors: Vega has 32 Itanium processors, each of which is roughly equivalent to the processor included with a new personal computer in power. In addition to the ability to simultaneouly execute 32 different jobs, these processors have the ability to manipulate especially large numbers efficiently because of their 64-bit architecture.
Memory: Vega has 128 GB of Random-Access Memory available, which is 64 times as much as even high-end desktop PCs sold in 2006. Applications that process vast amounts of data and store large intermediate results may not be capable of running without this quantity of memory. The way in which processors and memory units are interconnected is specifically designed to tasks or parts of tasks running on different processors to efficiently share memory. It should be noted that while most personal computers supplement their fast and expensive physical memory with larger quantities of slow and cheap swap space on disk. While vega does have approximately 40 GB of swap space available, this is not proportional to what users might expect. Thus, attempts to use significantly more than the total physical memory available may result in failure of tasks.
Disk: Users are typically allocated a small amount of disk space on vega's disks to be used for their own personal files. Additionally, their campus-wide AFS space is generally mounted for them. In addition to this user-specific space, there is a 274 GB /vtmp partition in which anyone may write files.
Software: LTS maintains a list of pre-installed software for scientific computing.
Vega and the other high-performance computing resources at Lehigh are (to the best of my knowledge) headless. This means that there is no specific display and keyboard associated with the computer. Rather, users are expected to remotely connect to these resources, providing input through their own local PC that will be forwarded to the desired computing resource. The LTS documentation how to setup this connection through the Secure Shell (SSH) protocol.
Vega runs a Linux kernel and (by default) uses the bash shell with which most user of UNIX-like operating systems should be familiar. Other shells are available, but if you intend to use one you probably do not need my help. The standard UNIX commands are available and quite useful -- emacs and vi to edit text documents, less to page text documents, grep to search text documents, ls to view the contents of a directory, and so on.
A lesser-known but remarkably useful command is screen. The idea of screen is to emulate a terminal. This has many, many uses, but I particularly suggest its use for starting a long-running process, detaching the screen from your PC, and re-attaching the screen to that PC or any other machine whenever you want to check on the progress of your process. The utility of this should be immediately obvious to anyone who has had a long-running process abort when their local machine crashed or their network connection dropped.
One final important command is nice. This command allows you to make optimal use of resources while being respectful of other users. There are 32 processors, but there are also generally more than 32 tasks that people would like to have running. The nice command allows you to make your task more willing to give up its fair share of resources other tasks. I generally run several long-running processes at once, one with a nice value of 0, another 2, another 4, and so on. This means all of my processes will run when no one else is competing with me for resources, but that I will not unfairly use 6 or 7 of the processors while others wait.
If you intend to use one of the standard software tools provided and supported by LTS, you can simply do so. However, it is common for users to run programs that they themselves have written, or that do not have a wide appeal. In these cases, it will usually be necessary to build programs on vega itself. This is unfortunately true because the Itanium architecture used by vega is different from the x86 architecture used by the LTS workstations and the Sparc architecture used by the CSE department workstations. Furthermore, you may find that libraries you link against may not exist on vega. Thus, it will be necessary to build these libraries in your home directory and use special directives to help the linker find them.
Good luck! If you have any further questions you might want to consult a good reference on Linux/UNIX use or send an email to the HPC mailing list.