RVIs are a significant cause of morbidity and mortality in immunocompromised children. We analyzed the characteristics and outcomes of infection by four respiratory viruses (RSV, adenovirus, influenza, and parainfluenza) treated at a pediatric tertiary care hospital in a retrospective cohort of patients who had received cancer chemotherapy, hematopoietic stem cell, or SOT. A total of 208 infections were studied among 166 unique patients over a time period of 1993–2006 for transplant recipients, and 2000–2005 for patients with cancer. RSV was the most common respiratory virus identified. There were 17 (10% of all patients) deaths overall, of which 12 were at least partly attributed to the presence of a RVI. In multivariate models, LRT symptoms in the absence of upper respiratory symptoms on presentation (OR 10.2 [2.3, 45.7], p = 0.002) and adenoviral infection (OR 3.7 [1.1, 12.6], p = 0.034) were significantly associated with poor outcome, defined as death or disability related to RVI. All of the deaths occurred in patients who had received either solid organ or HSCT. There were no infections resulting in death or disability in the cancer chemotherapy group.