Abstract

Varenicline, a widely used and successful smoking cessation agent, acts as a partial agonist at nicotinic acetylcholine receptors. Here, we explore the effects of varenicline at human and mouse 5-Hydroxytryptamine₃ (5-HT₃) receptors. Application of varenicline to human 5-HT₃ receptors expressed in Xenopus laevis oocytes reveal it is almost a full agonist (R_max = 80%) with an EC₅₀ (5.9 μM) 3-fold higher than 5-HT. At mouse 5-HT₃ receptors varenicline is a partial agonist (R_max = 35%) with an EC₅₀ (18 μM) 20-fold higher than 5-HT. Displacement of the competitive 5-HT₃ receptor antagonist [³H]granisetron reveals similar IC₅₀ values for varenicline at mouse and human receptors expressed in human embryonic kidney 293 cells, although studies in these cells using a membrane potential-sensitive dye show that again varenicline is a 4- or 35-fold less potent agonist than 5-HT in human and mouse receptors, respectively. Thus the data suggest that the efficacy, but not the affinity, of varenicline is greater at human 5-HT₃ receptors compared with mouse. Docking studies provide a possible explanation for this difference, because they suggest distinct orientations of the ligand in the mouse versus human 5-HT₃ agonist binding sites. Additional binding selectivity studies in a broad panel of recombinant receptors and enzymes confirmed an interaction with 5-HT₃ receptors but revealed no additional interactions of varenicline. Therefore, activation of human 5-HT₃ receptors may be responsible for some of the side effects that preclude use of higher doses during varenicline treatment.