To examine the association of low-grade systemic inflammation with diabetes, as well as its heterogeneity across subgroups, we designed a case-cohort study representing the ∼9-year experience of 10,275 Atherosclerosis Risk in Communities Study participants. Analytes were measured on stored plasma of 581 incident cases of diabetes and 572 noncases. Statistically significant hazard ratios of developing diabetes for those in the fourth (versus first) quartile of inflammation markers, adjusted for age, sex, ethnicity, study center, parental history of diabetes, and hypertension, ranged from 1.9 to 2.8 for sialic acid, orosomucoid, interleukin-6, and C-reactive protein. After additional adjustment for BMI, waist-to-hip ratio, and fasting glucose and insulin, only the interleukin-6 association remained statistically significant (HR = 1.6, 1.01–2.7). Exclusion of GAD antibody-positive individuals changed associations minimally. An overall inflammation score based on these four markers plus white cell count and fibrinogen predicted diabetes in whites but not African Americans (interaction P = 0.005) and in nonsmokers but not smokers (interaction P = 0.13). The fully adjusted hazard ratio comparing white nonsmokers with score extremes was 3.7 (P for linear trend = 0.008). In conclusion, a low-grade inflammation predicts incident type 2 diabetes. The association is absent in smokers and African-Americans.