ISE334

Course ISE334
Title Introduction to Multimedia Systems
Credits 3
Course Coordinator

Tony Scarlatos
Description

Survey of technologies available for user interfaces. Discussion of hypertext; voice, music, and video together with tools and models for capturing, editing, presenting, and combining them. Capabilities and characteristics of a range of peripheral devices including devices based on posture, gesture, head movement, and touch. Case studies of academic and commercial multimedia systems including virtual reality systems. Students participate in laboratory exercises and build a multimedia project. This course is offered as both CSE 334 and ISE 334.
Prerequisite Prerequisite: U2, U3 or U4 standing; CSE or ISE major
Course Outcomes
  • An understanding of multimedia standards and their perceptual limitations
  • An ability to understand and apply methods to digitize, process, compress, and encode multimedia data for a variety of delivery mechanisms
  • An ability to construct software systems for modeling, rendering and animating 3D objects
Textbook
  • Digital Multimedia, 2nd edition, Nigel and Jenny Chapman, ISBN: 0-470-85890-7, John Wiley & Sons, February 2004
Major Topics Covered in Course
  • What is Multimedia: Conduits of information: print, broadcast, internet, CD/DVD, Vehicles of expression: text, graphics, animation, audio, video, 3D, Multimedia systems approach.
  • The Nature of Information: Analog Events, Digital Recordings, Waveforms, Electromagnetic Spectrum, Color models, Frequency and Amplitude, Pitch and Volume, Color and Brightness, Transducers, Sampling rate, quantization.
  • The Biology and Psychology of Human Perception: Visible spectrum, Auditory range, Sensory bandwidth, Psychology of color, Pattern recognition.
  • The Design of Information: Aspect ratio and figure-ground relationships, Iconography, Design of statistical and schematic data, Cartography.
  • Intellectual Property : Copyright, Visual Artist's Rights Act, Trademark, Work for Hire.
  • The Future of "New Media": Immersive technology, Media convergence, Ubiquitous computing, Physical computing and multimodal interfaces, Computer-supported collaborative work.
  • Pictures: Image capture, resolution, Channels and layers, masking, Rasterizing, Graphics formats, Digital color and palettes, Image processing, Graphics compression.
  • Text and Graphics: Vectors, paths, Icons, Text formats, character sets, and fonts, Typography, Hypertext.
  • Audio: Audio capture, sampling rate and resolution, Digital audio vs. MIDI , Audio formats, Mixing and effects, Audio compression.
  • Animation: Animation principles, Frame rate, Rotoscopy, Interpolation, Kinematics, Animation formats.
  • 3D: 3D modeling, Motion capture, Texture and bump mapping, 3D formats, 3D animation.
  • Video : Video capture, frame rate and resolution, Analog video vs. digital video, Video formats and standards, Video compression.
Laboratory
  • Image capture; masking and alpha channel; image processing; compression; raster-to-vector conversion. (approx. 2 weeks)
  • IPaths; boolean operations; 2D extrusion to 3D; tessellation. (2 weeks)
  • Digital multitrack recording of live audio; mixing and loops; re-sampling; compression; MIDI . (2 weeks)
  • Frame-by-frame and interpolated animation; rotoscopy; morphing and spline-based animation. (2 weeks)
  • Spline and polygon mesh modeling; texture and bump mapping; lighting; morphing; particle systems; kinematics. (2 weeks)
  • Chroma and alpha channel video compositing; titling and transitions; compression; video streaming. (2 weeks)
Course Webpage

Crosslisted with CSE334