I don't remember if it's ever stated there, but I do think that after Season 4 the whole Holmes family looks rather in a bad light now.

His mother is a very caring and loving woman, who gave up her career to become a full-time mother. Though being a genius herself, it must have tested her parental skills to the utmost having to raise three children with IQ's of more than above average, while their EQ's were at only a minimum level.