For centuries, the history and music of Joseph Franz Haydn and Wolfgang Amadeus Mozart have been compared by scholars. Recently, the growing field of music information retrieval (MIR) has offered quantitative analyses to complement traditional qualitative analyses of these composers. In this MIR study, we classify the composer of Haydn and Mozart string quartets based on the content of their scores. Our contribution is an interpretable statistical and machine learning approach that provides high classification accuracies and musical relevance. We develop novel global features that are automatically computed from symbolic data and informed by musicological Haydn-Mozart comparative studies, particularly relating to the sonata form. Several of these proposed features are found to be important for distinguishing between Haydn and Mozart string quartets. Our Bayesian logistic regression model attains leave-one-out classification accuracies over 84%, higher than prior works and providing interpretations that could aid in assessing musicological claims. Overall, our work can help expand the longstanding dialogue surrounding Haydn and Mozart and exemplify the benefit of interpretable machine learning in MIR, with potential applications to music generation and classification of other classical composers.