The title of the following paper of David Goldberg says it all: What every computer scientist should know about floating-point arithmetic. We downloaded an edited reprint of ACM Computing Surveys 23(1):5-48, 1991 For the html version of this paper, click here.
Further recommended reading are the Lectures Notes of Professor W. Kahan on the status of the IEEE Standard 754. The standard itself is described (in pdf format) in IEEE Standard for Binary Floating-Point Arithmetic.