Advertisement
Research ArticleArticle

Microsomal Metabolism of Delavirdine: Evidence for Mechanism-Based Inactivation of Human Cytochrome P450 3A

Richard L. Voorman, Stephen M. Maio, N. Ann Payne, Zhiyang Zhao, Kenneth A. Koeplinger and Xiaohong Wang
Journal of Pharmacology and Experimental Therapeutics October 1998, 287 (1) 381-388;

Abstract

Administration of delavirdine, an HIV-1 reverse transcriptase inhibitor, to rats or monkeys resulted in apparent loss of hepatic microsomal CYP3A and delavirdine desalkylation activity. Human CYP3A catalyzes the formation of desalkyl delavirdine and 6′-hydroxy delavirdine, an unstable metabolite, while CYP2D6 catalyzes only desalkyl delavirdine. CYP2D6 catalyzed desalkyl delavirdine formation was linear with time (up to 30 min) but when catalyzed by cDNA expressed CYP3A4 or human liver microsomes the reaction rate declined progressively with time. Coincubation with triazolam showed that delavirdine caused a time- and NADPH-dependent loss of CYP3A4 activity in human liver microsomes as measured by triazolam 1′-hydroxylation. The catalytic activity loss was saturable and was characterized by aKi of 21.6 ± 8.9 μM and a kinact of 0.59 ± 0.08 min−1. An apparent partition ratio of 41 was determined with cDNA expressed CYP3A4, based on the substrate depletion method. Incubation of [14C]delavirdine with microsomes from several species resulted in irreversible association with an approximately 50 kDa protein, as demonstrated by SDS-PAGE/autoradiography. Binding to the protein was NADPH dependent, glutathione insensitive, proportional to the level of CYP3A expression and was inhibited by ketoconazole, a specific CYP3A inhibitor. NADPH-dependent irreversible binding to human and rat total microsomal protein was demonstrated following exhaustive extraction of microsomal protein. Binding was decreased in the presence of glutathione and appeared to be related to expression level of CYP3A. These results suggest that delavirdine can inactivate CYP3A and has the potential to slow the metabolism of coadministered CYP3A substrates.

The reverse transcriptase of HIV-1 catalyzes the transcription of viral RNA to proviral DNA, an essential step in the life cycle of HIV-1 and the progression to AIDS in humans (Rosenberg and Fauci, 1991). Inhibition of reverse transcriptase has been a significant target for therapeutic intervention in the disease and a number of nucleoside-based HIV-1 reverse transcriptase inhibitors, such as zidovudine, didanosine and lamivudine, have been shown effective in temporarily halting the progress of the disease (Declercq, 1994). Delavirdine (fig.1) (1-[3-[(1-methylethyl)amino]-2-pyridinyl]-4-[[5-[(methylsulfonyl)amino]-1H-indol-2-yl]carbonyl]-piperazine) is a potent, specific non-nucleoside inhibitor of HIV-1 reverse transcriptase (Dueweke et al., 1993) currently approved for the treatment of AIDS in combination with other antiretroviral agents.

Figure 1
Figure 1

Structure of delavirdine.

Analysis of plasma drug levels in male rats treated orally or intravenously with single doses of delavirdine showed that the drug was well absorbed; however, clearance of delavirdine was diminished and half-life was increased in rats treated with increasing doses of delavirdine (Adams et al., 1996). Studies on the excretion of radiolabeled delavirdine from rats indicate is cleared from plasma primarily by metabolism and was apparently eliminated more rapidly from male rats than female rats (Chang et al., 1997). Human clinical studies on delavirdine have shown evidence for capacity-limited clearance and inhibition of CYP3A (Cox et al., 1993; Cheng et al., 1997).

The in vitro metabolism of delavirdine was examined using liver microsomes from several species (Voorman et al., 1998). Microsomal metabolic profiles among all species, including human, differed only by the relative concentrations of the major metabolites, desalkyl delavirdine and 6′-hydroxy-delavirdine. The exact structure of the latter was not clear but appeared to be unstable and slowly degraded, probably through loss of the pyridine functionality. The KM for liver microsomal desalkylation across several species, ranged from 4.4 to 12.6 μM. Delavirdine desalkylation by microsomes pooled from several human livers was characterized by an apparent KM of 6.8 μM and Vmax of 0.44 nmol/min/mg. Delavirdine was metabolized primarily by CYP3A, which catalyzed both delavirdine desalkylation and 6′-hydroxylation; the apparentKM for desalkylation by CYP3A4 was 5.4 μM. Delavirdine was also metabolized by CYP2D6, which catalyzed only desalkylation, characterized by an apparentKM of 10.9 μM and was probably a lower capacity pathway than CYP3A.

In this report we show that delavirdine appears to be metabolically activated by CYP3A, forming a reactive but unidentified metabolite that binds irreversibly to the enzyme. Accordingly, this conclusion coupled with the observation that delavirdine plasma concentrations in humans can exceed the observed KM for microsomal delavirdine desalkylation imply that in vivo clearance of delavirdine will be likely be attenuated as a consequence of catalytic saturation and diminished enzymatic capacity.

Materials and Methods

Chemicals.

Triazolam, 1′-hydroxytriazolam, U-88822 and desalkyl delavirdine were prepared by Medicinal Chemistry Research, Pharmacia and Upjohn (Kalamazoo, MI). cDNA-expressed human CYP2D6 and CYP3A4 (expressed in a lymphoblastoid cell line) were purchased from Gentest (Woburn, MA). NADP+, isocitrate dehydrogenase, trisodium isocitrate and testosterone were obtained from Sigma Chemical (St. Louis, MO). [2-14C-Pyridine]delavirdine mesylate, 1-[3-[(1-methylethyl)amino]-2-pyridinyl]-4-[[5-[(methylsulfonyl)amino]-1H-indol-2-yl]carbonyl]piperazine, monomethanesulfonate, 79.58 μCi/mg, radiochemical purity >99.9% by HPLC and TLC was synthesized by J.A. Easter, Drug Metabolism Research, Pharmacia & Upjohn. All other supplies were obtained as highest reasonable purity from standard supply houses.

Liver microsomes.

Transplant quality human liver tissue, perfused with Belzer’s solution, was obtained through the International Institute for the Advancement of Medicine (Exton, PA). Tissue was flash frozen within 24 hr of removal from donor and stored at –80° to –190°C. Microsomes were prepared essentially as described (Lu and Levin, 1972) and stored in 0.25 M sucrose at –80°C. Pooled human liver microsomes were prepared both by blending microsomal samples prepared from individual livers or by homogenizing several liver samples, blending the homogenates and completing the microsomal isolation with the blended homogenates.

Male Sprague-Dawley rats, weighing ∼250 g, were obtained from Charles River (Portage, MI). Livers from untreated rats were used for control microsomal activity. For selective induction of CYP3A, rats were treated with daily oral doses of PCN (100 mg/kg; suspension in corn oil, 100 mg/ml) for 3 days and killed 24 hr after last dose. For determination of the effect of delavirdine treatment on rat microsomal cytochrome P450, rats were administered twice daily oral doses of delavirdine mesylate (acidic solution in 30% propylene glycol) for 29 days and killed ∼16 hr after last dose. Male or female beagle dogs were obtained from Marshall Farms (North Rose, NY) and used without further treatment. Male CD-1 mice were obtained from Charles River; male and female cynomolgus monkeys were obtained from the Pharmacia & Upjohn primate colony. Female monkeys were administered twice daily doses of delavirdine mesylate (acidic solution in 80% propylene glycol) for 29 days and killed ∼16 hr after the last dose. Following animal euthanasia, livers were perfused with cold saline, frozen in liquid nitrogen and stored at –80°C. Hepatic microsomes were prepared by differential centrifugation of liver homogenates, based on standard methods (vanderHoeven and Coon, 1974). All tissue manipulations were conducted at 4°C. Liver tissue was homogenized in 4 vol 1.15% KCl, 10 mM EDTA, pH 7.4 using a motor-driven Teflon mortar-glass pestle homogenizer and centrifuged at 10,000 ×g for 20 min; the supernatant was further centrifuged at 227,000 × g for 40 min. The resulting microsomal pellet was washed by homogenization in 100 mM sodium pyrophosphate, pH 7.4, 1 mM EDTA, and pelleted by centrifugation at 227,000 ×g for 40 min. The microsomal pellet was finally homogenized in 0.25 M sucrose, 0.1 mM EDTA and stored at –80°C.

Assays.

Protein was determined by the bicinchoninic acid assay using a microtitre plate format (Redinbaugh and Turley, 1986) and standardized relative to bovine serum albumin. Total cytochrome P450 content was determined spectrophotometrically by measuring absorption of the dithionite-reduced CO difference spectrum and using E450–490 = 91 mM−1cm−1 (Omura and Sato, 1964). Microsomal erythromycin N-demethylation activity was assayed essentially as described (Prough and Ziegler, 1977)

Metabolism of delavirdine was determined with 0.5 to 1 mg/ml microsomal protein, 50 mM potassium phosphate or HEPES buffer, pH 7.4, 0.1 mM EDTA at 37°C. Microsomal suspensions were diluted in buffer (final vol 0.1–0.5 ml) followed by addition of drug in methanol (1% final conc.). The reaction was started by addition of an NADPH generating system consisting of (final conc.) 1 mM β-NADP+, 5 mM trisodium isocitrate, 5 mM magnesium chloride, and 0.4 units/ml isocitrate dehydrogenase and quenched after 3 min by addition of an equal volume of acetonitrile containing an internal standard (U-88822). Delavirdine and desalkyl delavirdine were measured by reversed phase HPLC using gradient or isocratic elution. For gradient elution a YMC Basic column, 4.6 mm i.d. × 25 cm (YMC, Inc., Wilmington, NC), was used along with a mobile phase (1 ml/min) consisting of 10% acetonitrile for 5 min followed by a linear gradient over 35 min to 60% acetonitrile and finally a 5 min hold at 60% acetonitrile; the gradient was balanced to 100% with 0.1 M ammonium acetate, pH 4, containing 3% acetonitrile. For isocratic elution a Zorbax SB-CN column, 4.6 mm i.d. × 15 cm (MAC-MOD Analytical, Inc., Chadds Ford, PA) was used with a mobile phase consisting of 26% acetonitrile and 74% 50 mM ammonium acetate, pH 5.8, containing 3% acetonitrile, flowing at 1.5 ml/min. Detection was by UV absorbance at 295 nm. Peak height response data were collected and analyzed by an in-house chromatography data system.

Testosterone 6β-hydroxylation, an index of CYP3A activity, was measured by minor modification of established methods (Reinerinket al., 1991). The reaction mixture consisted of liver microsomes (0.4 mg/ml), potassium phosphate (50 mM, pH 7.4), EDTA (0.1 mM), NADPH generating system (see above) and 500 μM testosterone in a final volume of 0.5 ml. The reaction was started, following 3 min preincubation at 37°C, by addition of the NADPH generating system and allowed to proceed for 10 min or as described. Reactions were terminated by addition of 50 μl acetic acid; samples were vortexed, placed on ice for 10 min and briefly centrifuged to pellet microsomal protein. Supernatants were assayed for testosterone metabolites by HPLC. The HPLC system consisted of a Perkin-Elmer ISS 100 autosampler, a model 410 pump (Perkin Elmer, Norwalk, CT) and an Anspec (Ann Arbor, MI) Spectromonitor D UV detector set at 254 nm. Metabolites were separated on a 250 × 4.6 mm YMC ODS-AQ column (Wilmington, NC) enclosed in a column oven set at 50°C, using gradient elution (1 ml/min). The mobile phase consisted of methanol/acetonitrile (90:10) 0.5% acetic acid (A) and methanol/water (30:70) 0.5% acetic acid (B); the gradient consisted of 10% A for 10 min, followed by a linear gradient to 28% A over 12 min, another linear gradient to 45% A over 5 min, then holding 45% A for 5 min. Concentration of 6β-hydroxy testosterone was determined by comparison to a standard curve.

Microsomal triazolam 1′-hydroxylation, an index of CYP3A activity, was measured by a modified published method (Kronbach et al.,1989). Incubation mixtures consisted of liver microsomes in 50 mM potassium phosphate buffer, pH 7.4, 0.1 mM disodium EDTA and an NADPH generating system (see above). The reaction was started by addition of NADPH and allowed to proceed for 20 min. Preliminary incubations showed product formation to be linear over this reaction period. Reactions were terminated by addition of an equal volume of cold acetonitrile/methanol (8:2, v/v). Supernatants were assayed for triazolam metabolites (1′-OH and 4-OH) by HPLC using isocratic elution. The HPLC system consisted of a Perkin-Elmer 410 pump and Kratos (Anspec, Ann Arbor, MI) Spectraflow 773 UV detector set at 295 nm. Metabolites were separated on a 150 × 4.6 mm Zorbax SB-C18 column using a mobile phase consisting of 40% methanol, 5% acetonitrile and 55% 10 mM potassium phosphate, pH 7.4, and flowing at 1 ml/min.

In order to characterize the time dependence of delavirdine inhibition of CYP3A, human liver microsomes (0.5 μM cytochrome P450) were incubated in the presence of NADPH with varying concentrations of delavirdine. At specific times after initiation of the incubation, aliquots (50 μl) were withdrawn and diluted 20-fold into a prewarmed (37°C) incubation system containing NADPH and triazolam (100 μM) in 50 mM phosphate buffer, pH 7.4 and allowed to incubate a further 20 min. Reactions were quenched as described above and assayed for 1′-hydroxy triazolam.

The partition ratio of delavirdine and CYP3A4 was determined by the substrate depletion method. CYP3A4 (0.1 mg) was incubated with delavirdine (4 or 10 μM) along with the NADPH generating system in 50 mM potassium phosphate, pH 7.4, 0.1 mM EDTA in a final volume of 0.1 ml. The reaction was quenched after 45 or 90 min and assayed for delavirdine content. Samples were incubated in triplicate.

SDS-PAGE/Autoradiography of protein reactive products.

Gel dimensions for SDS-PAGE were 14 × 16 × 0.15 cm (Hoefer Model SE 400; Hoefer Scientific, San Francisco, CA) and consisted of a 4.5% acrylamide stacking gel and 10% acrylamide running gel (Laemmli, 1970). Liver microsomes (0.2 mg), 50 μM [14C]delavirdine, an NADPH generating system and 50 mM HEPES buffer, pH 7.4 containing 0.1 mM EDTA in a total volume of 0.2 ml. Additionally, 5 mM glutathione or 10 μM ketoconazole were included with certain samples while NADPH was omitted from others. Incubations were at 37°C for 20 min and terminated by addition of 200 μl 2× sample buffer, boiled for 5 min and ∼75 μg microsomal protein was loaded into each well. Gels were run under reducing conditions using α-mercaptoethanol and were run at 30 mA. [14C]Methylated-protein molecular weight markers (Amersham, Arlington Heights, IL) were also run. Gels were fixed in 30% methanol, 10% acetic acid, treated with ENHANCE (Dupont-NEN, Boston, MA), and dried. Dried gel was exposed to X-ray film (Kodak XAR 5) along with an intensifying screen at –70°C for 4 days and developed.

Irreversible binding of [14C]delavirdine to microsomal proteins.

Liver microsomes (3 mg) were incubated with 100 μM [2-14C-pyridine]delavirdine in 50 mM HEPES, pH 7.4 containing 0.1 mM EDTA and NADPH generating system (see above) in a final volume of 1 ml at 37°C in a reciprocating water bath. Certain samples contained 30 μM ketoconazole, 5 mM glutathione or 100 μM nonradiolabeled delavirdine and NADPH was omitted from others. Reactions were started by addition of the NADPH generating system, allowed to proceed for 35 min and quenched by addition of 5 ml ice-cold methanol. Following centrifugation the protein pellet was dissolved in 0.5 ml 1% SDS, vortexed vigorously and reprecipitated by addition of 3 ml ice-cold methanol (Sadrieh and Thomas, 1994). The washing steps were repeated two more times in order to wash-out entrapped or unbound radioactivity. Pellets were dried overnight, dissolved in 500 μl 1% SDS, and 400 μl was added to 15 ml Ultima Gold cocktail (Beckman Instruments) and assayed by liquid scintillation counting. Portions of supernatants from the various washing steps were also assayed for radioactivity to monitor washing efficiency. An aliquot of the final pellet solution (10 μl) was assayed for protein.

Data analysis.

For analysis of time and concentration dependent inhibition data according to the model set forth by Kitz and Wilson (Kitz and Wilson, 1962), first order rate constants for inhibition were estimated by nonlinear regression of a single exponential curve using PRISM version 2.0 (GraphPAD Software, San Diego, CA). Kinetic parameters, Ki andkinact, for delavirdine inhibition were estimated by nonlinear regression analysis of the hyperbolic equation:ki=kinact I/(Ki+I) where ki represents the estimated rate constant for inhibition and I represents the corresponding delavirdine concentration. For evaluation of desalkyl delavirdine interaction with CYP3A, estimation of theKI was done using the program ENZYME KINETICS (Trinity Software, Campton, NH). Statistically significant differences between treated and control rats was determined by analysis of variance using Dunnett’s test at the 5% level (SAS version 6.07; SAS Institute, Cary, NC).

Results

Hepatic microsomal activity in delavirdine treated animals.

Hepatic microsomes isolated from drug-treated rats showed a substantial decline in erythromycin N-demethylation (4-fold), a reporter assay for CYP3A activity (table 1). Analysis of delavirdine desalkylation showed that it paralleled the loss in CYP3A activity with increasing dose. Microsomal samples were assayed for residual delavirdine and its major metabolite; less than 5 μM delavirdine and less than 0.1 μM desalkyl delavirdine were observed remaining in the high dose samples. A mean of 1.08 μM delavirdine was found in the lower dose samples, which nevertheless showed a significant loss of CYP3A activity. Similar results were observed in microsomes from female cynomolgus monkeys treated with up to 350 mg/kg/day delavirdine for 4 weeks. A 10-fold loss of testosterone 6β-hydroxylation activity and 5-fold loss of delavirdine desalkylation was observed in the high dose group (table2).

View this table:
Table 1

Evaluation of microsomal cytochrome P450 in rats treated for 4 weeks with delavirdine mesylate

View this table:
Table 2

Effect of delavirdine treatment of microsomal cytochrome P450 in monkeys

Progressive inhibition of delavirdine metabolism.

Measurement of desalkyl delavirdine formation upon incubation of delavirdine with human liver microsomes and NADPH revealed a very short period of linearity, showing instead a progressive decline in delavirdine desalkylation (fig. 2). The decreasing rate was not due to falling substrate concentration since delavirdine remained above 30 μM (not shown) and has an apparentKM of 6.8 μM (Voorman et al., 1998). Likewise, metabolism of delavirdine by cDNA expressed CYP3A4 resulted in a similar decreasing reaction rate over time. On the other hand, measurement of desalkyl delavirdine formation by cDNA expressed CYP2D6 showed linear product formation for the 30 min duration of the assay (fig. 2).

Figure 2
Figure 2

Time course of microsomal delavirdine metabolism by human liver microsomes or cDNA expressed CYP2D6 or 3A4. Delavirdine (30 μM) was incubated with human liver microsomes (0.5 mg/ml) or cDNA expressed cytochrome P450 isoforms (1 mg/ml). At prescribed times, aliquots were withdrawn, quenched, and assayed for desalkyl delavirdine. Left, cumulative formation of desalkyl delavirdine. Right, incremental reaction rate which was estimated by the product formed during each sampling interval.

The time course of triazolam metabolism by human liver microsomes produced moderately linear product formation, however, the rate was obviously attenuated in a nonlinear manner when coincubated with 10 μM delavirdine (fig. 3). Similar results were found in an evaluation of testosterone metabolism that showed linear formation of 6β-hydroxy testosterone but when coincubated with 10 μM delavirdine showed the same progressive loss of metabolic capacity (fig. 3). The concentration of testosterone was 500 μM, ∼10-fold over apparent KM for 6β-hydroxylation (Miyata et al., 1994), while the concentration of delavirdine was less than 2-fold over theKM for CYP3A-dependent delavirdine desalkylation.

Figure 3
Figure 3

Effect of coincubation of delavirdine with CYP3A substrates. Triazolam (30 μM) was incubated with human liver microsomes (0.2 mg/ml) along with NADPH either with or without 10 μM delavirdine and assayed for cumulative formation of 1′-hydroxytriazolam (top). Testosterone (500 μM) was incubated with human liver microsomes (0.1 mg/ml) along with NADPH either with or without 10 μM delavirdine and assayed for cumulative formation of 6β-hydroxy testosterone. DLV refers to delavirdine.

Kinetic evaluation of inactivation.

The potential for delavirdine to inactivate CYP3A was evaluated by incubation of delavirdine with human liver microsomes followed by dilution and assay of residual CYP3A activity. Results showed an exponential (pseudo first order) loss of CYP3A activity as a function of time and showed that the rate of inactivation moved toward saturation with increasing delavirdine concentration (fig. 4), highly suggestive of mechanism-based inhibition. CYP3A activity loss, as reported by triazolam hydroxylation, was modeled as a saturable process and was characterized by a Ki of 21.6 ± 8.9 μM and a kinactof 0.59 ± 0.08 min−1. The latter may be considered analogous to Vmax; in other words, the maximum pseudo first order inactivation rate constant. The parameter kinact can be converted to a half-life, which for the enzymatic conditions of this experiment would be equal to 1.17 min. Inspection of an Eadie-Hofstee plot revealed a poor fit, owing largely to what might be a non-pseudo first order rate constant derived from the 100 μM delavirdine incubation. Elimination of this point afforded a better fit, yielding aKi of 9.48 ± 1.66 μM and akinact of 0.44 ± 0.02 min−1. In this case theKi is closer to the apparentKM (6.8 μM) (Voorman et al.,1998) for human microsomal metabolism of delavirdine; others have shown that Ki should approximateKM (Lopez Garcia et al., 1994). Significant inhibition was observed in the zero-time samples, probably a result of residual delavirdine since the initial incubation was diluted only 20-fold to retain sufficient sensitivity for triazolam 4′-hydroxylation.

Figure 4
Figure 4

Delavirdine concentration and time dependency for inactivation of CYP3A as reported by triazolam 1′-hydroxylation. Delavirdine was incubated, at the concentrations shown, with human liver microsomes (0.5 μM cytochrome P450) in the presence of NADPH. At specified times aliquots of the incubation were diluted 20-fold (0.025 μM cytochrome P450), fortified with 100 μM triazolam and NADPH, incubated for a further 20 min and assayed for residual CYP3A activity using triazolam 1′-hydroxylation as a CYP3A specific reporter assay. Top, delavirdine concentrations and remaining CYP3A activity. Inactivation rates were determined by nonlinear fit and rates were fitted against inhibitor concentration using a nonlinear fit. Bottom, plot of inactivation rate with increasing delavirdine concentration.

Effect of desalkyl delavirdine on CYP3A activity.

The influence of desalkyl delavirdine on CYP3A was determined by measuring triazolam 1′-hydroxylation in the presence of 5 or 20 μM desalkyl delavirdine. An inhibitory effect was observed which showed aKI of 31 μM (fig.5).

Figure 5
Figure 5

Evaluation of desalkyl delavirdine as an inhibitor of human liver microsomal triazolam 1′-hydroxylation. Human liver microsomes (0.05 μM cytochrome P450) were incubated with varying concentrations of triazolam (2–271 μM) in the presence of NADPH and varying concentrations of desalkyl delavirdine. Reactions were quenched after 15 min and assayed for triazolam 1′-hydroxylation.

Partition ratio.

The partition ratio was determined by the substrate depletion method. Delavirdine was incubated with CYP3A4, shown by the supplier to contain 95 pmol CYP3A4 per mg protein, for 90 min, sufficient time for nearly complete inactivation of the enzyme. Sampling at 45 min and 90 min showed only 10% further substrate consumption between 45 and 90 min. By simple arithmetic determination, based on substrate consumed by a defined amount of CYP3A4, to the point of presumed complete inactivation, yielded a partition ratio of 41. This would suggest the enzyme turns over ∼41 moles of substrate for every mole of enzyme inactivated.

SDS-PAGE analysis of irreversible binding.

The likelihood of irreversible association of delavirdine with CYP3A was tested by incubation of [14C]delavirdine with microsomes from mouse, dog, monkey, rat and human followed by SDS-PAGE/autoradiography analysis (fig.6). Among all tested species there was clearly an irreversible association of radioactivity with an ∼50-kDa protein, very likely representing one or more cytochrome P450 isoforms and reflecting reaction with at least one specific protein. Binding to protein was NADPH-dependent, consistent with the need for metabolic activation. There was distinctly greater binding to protein from PCN treated rats compared to untreated rats. The same was true for a human microsomal sample with high testosterone 6α-hydroxylation activity (HLM 18; 11.7 nmol/min/mg) compared to a human sample with lower activity (HLM 12; 5.1 nmol/min/mg). Addition of ketoconazole inhibited binding in the human sample. Addition of glutathione had little apparent effect on the binding, indicating either unreactivity of the activated species with glutathione or inability of glutathione to gain access to the active site.

Figure 6
Figure 6

SDS-PAGE autoradiography analysis of microsomal samples incubated with [14C]delavirdine. Microsomal samples were incubated with [14C]delavirdine (50 μM) for 20 min. Reactions were quenched by addition of SDS-PAGE sample buffer and analyzed by SDS-PAGE autoradiography. UT Rat refers to microsomes from untreated male rat; PCN Rat refers to microsomes isolated from rats treated with pregnenolone I-carbonitrile, a CYP3A inducer.

Irreversible association of radioactivity with microsomal protein.

Total binding of delavirdine derived radioactivity to microsomal protein was evaluated using microsomal preparations from rats and humans (fig. 7). Protein-reactive binding was NADPH dependent, indicating the need for metabolic activation in the form of either reduction or catalysis. Binding was greatly increased in the rat and human samples with higher activities of CYP3A and was reduced by addition of ketoconazole, all pointing to a catalytic role for CYP3A in the binding process. Radiolabel binding was reduced by approximately half when incubated with equimolar unlabeled delavirdine, supporting the notion of enzymatic involvement in activation of delavirdine to a protein reactive entity. Addition of glutathione to the incubation reduced the level of binding to total microsomal protein.

Figure 7
Figure 7

Irreversible association of radioactivity with microsomal protein from incubation of [14C]delavirdine with microsomal protein. Microsomal samples were incubated with [14C]delavirdine; protein precipitates were washed, solubilized and assayed for radioactivity and protein. PCN Rat refers to microsomes isolated from rats treated with pregnenolone α-carbonitrile, a CYP3A inducer; Keto refers to added ketoconazole; DLV refers to added delavirdine; GSH refers to added glutathione.

Discussion

The results of the present set of experiments support the notion that delavirdine is a mechanism-based inhibitor of CYP3A4 (Silverman, 1988). Triazolam 4′-hydroxylation was inhibited in a time- and concentration-dependent manner by delavirdine and showed a trend toward saturation of the inactivation rate. Association of nonextractable radioactivity with microsomal protein showed the irreversibility of the reaction. Substrate protection was shown by the ability of ketoconazole to block irreversible binding of delavirdine to microsomal protein. Catalytic dependency of the inactivation reaction was shown by the lack of irreversible binding to microsomal protein in the absence of NADPH and by the ability of ketoconazole to block the reaction.

Circumstantial evidence for delavirdine dependent inhibition of CYP3A may be found in the analysis of CYP3A activity in delavirdine treated rats and monkeys. CYP3A activity was reduced 4- to 10-fold in microsomes isolated from delavirdine-treated rats and monkeys; however, the reduced activity did not appear to be a consequence of residual delavirdine or its major metabolite. Although these results could be explained by a decline in CYP3A expression, unpublished experiments on evaluation of rat microsomal CYP3A content showed no loss of CYP3A in delavirdine treated rats. Similarly, human clinical studies have described an apparent loss of CYP3A capacity during treatment with delavirdine (Cheng et al., 1997).

Product inhibition was ruled out by showing a relatively weak interaction of desalkyl delavirdine with CYP3A. Although desalkyl delavirdine might contribute to the observed apparent autoinhibitory effects, the measured product concentrations (fig. 2 and table 1) would be insufficient to produce the degree of inhibition observed. Furthermore, human clinical use of delavirdine produced significant inhibition of CYP3A (Cheng et al., 1997) yet plasma levels of desalkyl delavirdine were ∼10-fold lower (3 μM) than the apparent Ki (31 μM) for inhibition of CYP3A. While these data do not rule out the influence of other unknown metabolites, it seems unlikely that desalkyl delavirdine could be responsible for the observed delavirdine dependent inhibition of CYP3A.

The mechanism of the reaction is not clear and the key reactive intermediate unknown. Since CYP2D6 catalyzed delavirdine desalkylation, along with CYP3A4, but was not apparently inhibited or inactivated by the reaction, desalkyl delavirdine is not likely involved in the inactivation; accordingly, the 6′-hydroxy metabolite or other unobserved CYP3A4 dependent metabolites could be implicated as the basis for inactivation. The involvement of 6′-hydroxydelavirdine could possibly be addressed through use of 6′-deuterodelavirdine. If indeed 6′-hydroxylation is the reactive path, then a kinetic isotope effect should be observed for the inactivation rate. Although the 6′-hydroxy metabolite degrades with subsequent cleavage of the pyridine-piperazine bond, the molecule does not appear to be highly electrophilic as might be expected of a reactive species. No glutathione or cysteine conjugates of delavirdine or its metabolites have been observed in animal or human metabolism studies. Apart from involvement of the 6′-hydroxydelavirdine, implication of any other reactive species would be highly speculative, since no other in vitro or in vivo metabolites have been observed which do not apparently derive from the two pathways described (there was also a very minor amidase pathway (Chang et al., 1997). Other reactive metabolites which could be formed from delavirdine might include an iminium ion derived from the piperazine amino group (Gorrod and Aislaitner, 1994) or other reactive species derived by way of a pyridine N-oxide (Damaniet al., 1982) or pyridine oxaziridine (Desai et al., 1993). It is likely that the reactive metabolite includes the entire three-ring structure of delavirdine since irreversible binding was observed with either pyridine or carboxamide labeled delavirdine (not shown).

Certain mechanism-based inhibitors, such as 17α-acetylenic steroids, are thought to covalently bind with cytochrome P450 heme and do not show detectable protein associated radioactivity since cytochrome P450 heme should be dissociated from the protein under conditions of SDS-PAGE (Guengerich, 1990). Thus, the present autoradiography results suggest that some portion of delavirdine probably binds to the cytochrome P450 apoprotein; however, binding to heme cannot be ruled out based on these experiments. Similarly, certain mechanism based inhibitors bind in the active site of cytochrome P450 in such a way as to interfere with CO binding (Chiba et al., 1995; Kunze and Trager, 1993) however, incubation of rat or human liver microsomes with delavirdine resulted in no obvious change in CO binding to cytochrome P450 heme (not shown). Indeed, no unusual spectral interactions between delavirdine and microsomal cytochrome P450 were observed (not shown). The nature of the binding and reaction site(s) will require further experimentation.

The derived inhibition parameter values are comparable to those of other suicide or mechanism-based cytochrome P450 inhibitors such as propranolol, furafylline, and gestodene (Masubuchi et al.,1994; Kunze and Trager, 1993; Guengerich, 1990). Delavirdine appears kinetically similar to gestodene, a CYP3A4 inhibitor, which displayed aKi of 46 μM and akinact of 0.39 min−1 but which showed a partition ratio of 9. Furafylline, an inhibitor of CYP1A2, was characterized by aKi of 23 μM and akinact of 0.87 min−1 in one report (Kunze and Trager, 1993) while another showed a Ki of 6.9 μM (Tassaneeyakul et al., 1994) and illustrating the potential for variation in the determination of pseudo first order rate constants. Tienilic acid inhibition of CYP2C9 was demonstrated with aKi of 4.3 μM andkinact of 0.21 min−1 (Lopez Garcia et al.,1994). The thiosteroid, spironolactone, inactivated rat CYP3A with aKi of 17 μM,kinact of 0.08 min−1 and a partition ratio of ∼22 (Decker et al., 1989). Interestingly, the furanopyridine based HIV-1 protease inhibitor L-754,394 was shown to be a very potent mechanism-based inhibitor of CYP3A4 with aKi of 7.5 μM andkinact of 1.62 min−1, not greatly different from delavirdine but with a very low partition ratio of 1.35 (Chiba et al., 1995).

Delavirdine appears to be analogous to propranolol in its effects on microsomal metabolism and subsequent altered plasma pharmacokinetics. Incubation of propranolol with rat liver microsomes results in irreversible association of propranolol with microsomal protein and implicated CYP2D in the activation process (Masubuchi et al., 1994). Multiple dose treatment of rats with propranolol resulted in reduced plasma clearance compared to a single dose, which could not be explained by simple saturation kinetics, suggesting an alteration in enzyme activity (Weber et al., 1994). Other studies have shown irreversible binding to microsomal protein in propranolol-treated rats (Schneck and Pritchard, 1981).

In addition to propranolol, irreversible protein binding has been observed with several other drugs approved for human use. Microsomal metabolism of the cholinesterase inhibitor, tacrine, produces irreversible protein binding with human or rat liver microsomes (Woolfet al., 1993). Tamoxifen, used in the treatment of breast cancer, was apparently metabolized by CYP3A from rat and human liver microsomes to an entity which resulted in irreversible association with microsomal protein (Mani et al., 1994). Cyclosporin A was shown to be activated by rat liver microsomes to produce irreversible protein adducts (Sadrieh and Thomas, 1994).

Relating the in vitro results to in vivopharmacokinetics is not straight forward. The in vitrometabolism system used here was a static system: unchanging enzyme and no product removal. In contrast an in vivo system is dynamic: enzyme is continually synthesized and added to the system with zero order kinetics (constant input), as a consequence of normal protein turnover (Correia, 1991), and metabolites are removed by circulation and/or conjugation systems. Thus, the degree of tissue binding and enzyme inhibition observed in these experiments might not be obtained during animal or human metabolism of delavirdine.

Present clinical use of delavirdine involves a dosing regimen which can produce average nadir plasma concentrations in the range of 10 μM delavirdine (Cheng et al., 1997). Since hepatic drug levels will likely be several fold greater than plasma, delavirdine levels in the liver could exceed the apparent Ki for inhibition of CYP3A, leading to enzyme inhibition. Evaluation of multiple-dose studies of delavirdine in humans showed increasing plasma delavirdine concentrations and a decline in the ratio of plasma desalkyl delavirdine to delavirdine over time, pointing to a loss of metabolic capacity (Cheng et al., 1997).

Although delavirdine is an apparent CYP3A4 inhibitor, as evidenced by strong inhibition of the erythromycin breath test (Cheng et al., 1997), it is also highly susceptible to increased clearance as a result of CYP3A4 induction by rifampicin (Borin et al.,1997). These discordant clinical observations might be explained by the relatively high partition ratio observed for delavirdine. The apparent clinical pharmacokinetic effect of a mechanism-based inhibitor would be a function of its Ki ,kinact and partition ratio and the zero-order synthesis rate of new or replacement enzyme. Thus, for delavirdine under basal conditions it would appear that inactivation of the enzyme exceeds synthesis of new enzyme (CYP3A4) but under circumstances of rifampicin induction, enzyme synthesis exceeds inactivation and the drug exhibits attributes of high clearance. Although rifampicin also induces P-glycoprotein and could produce lower net drug absorption (Schuetz et al., 1996) evaluation of plasma delavirdine metabolite formation indicates increased metabolism rather than decreased net absorption (Borin et al., 1997).

In conclusion, delavirdine appears to be metabolically activated by CYP3A, forming a reactive but unidentified metabolite which then binds irreversibly to CYP3A, destroying its catalytic ability. Accordingly, this conclusion coupled with the fact that delavirdine plasma concentrations in humans can exceed the observed microsomalKM and Ki imply that in vivo clearance of delavirdine will be likely be attenuated as a consequence of catalytic saturation and diminished enzymatic capacity.

Acknowledgments

The authors wish to thank R. S. P. Hsi, J. A. Easter and E. H. Chew for synthesis of radiolabeled delavirdine, C. C. Ratke for treatment of rats and R. K. Jensen for treatment of monkeys. We also thank L. C. Wienkers and R. C. Steenwyk for helpful discussions and gratefully acknowledge the help of J. Lord in preparation of the manuscript.

Footnotes

  • Send reprint requests to: Dr. Richard L. Voorman, Drug Metabolism Research B 300–3, Pharmacia and Upjohn, 301 Henrietta St., Kalamazoo, MI 49007. E-mail: Richard.L.Voorman{at}am.pnu.com

  • ↵1 This work was presented in part at the 9th International Conference on Cytochrome P450, Zurich, Switzerland, 1995.

  • Abbreviations:
    HIV-1
    human immunodeficiency virus type-1
    AIDS
    acquired immune deficiency syndrome
    API
    atmospheric pressure ionization
    ELISA
    enzyme linked immunosorbent assay
    MS
    mass spectrometry
    PCN
    pregnenolone 16α-carbonitrile
    SDS-PAGE
    sodium dodecyl sulfate-polyacrylamide gel electrophoresis
    • Received March 5, 1998.
    • Accepted May 19, 1998.

References

  1. ↵
    1. Adams WJ,
    2. Hosley JD,
    3. Sedlock ML,
    4. Anstadt RA,
    5. Romero DL,
    6. Morozowich W,
    7. Smith DP,
    8. Douglas SL
    (1996) Pharmacokinetics, oral bioavailability and dose proportionality of the HIV-1 reverse transcriptase inhibitor delavirdine in the male Sprague-Dawley rat. ISSX Proc 10:428.
    OpenUrl
  2. ↵
    1. Borin MT,
    2. Chambers JH,
    3. Carel BJ,
    4. Gagnon S,
    5. Freimuth WW
    (1997) Pharmacokinetic study of the interaction between rifampin and delavirdine mesylate. Clin Pharmacol Ther 61:544–553.
    OpenUrlCrossRefPubMed
  3. ↵
    1. Chang M,
    2. Sood VK,
    3. Wilson GJ,
    4. Kloosterman DA,
    5. Sanders PE,
    6. Hauer MJ,
    7. Fagerness PE
    (1997) Metabolism of the human immunodeficiency virus type 1 reverse transcriptase inhibitor delavirdine in rats. Drug Metab Dispos 25:228–242.
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Cheng CL,
    2. Smith DE,
    3. Carver PL,
    4. Cox SR,
    5. Watkins PB,
    6. Blake DS,
    7. Kauffman CA,
    8. Meyer KM,
    9. Amidon GL,
    10. Stetson PL
    (1997) Steady-state pharmacokinetics of delavirdine in HIV-positive patients: effect on erythromycin breath test. Clin Pharmacol Ther 61:531–543.
    OpenUrlCrossRefPubMed
  5. ↵
    1. Chiba M,
    2. Nishime JA,
    3. Lin JH
    (1995) Potent and selective inactivation of human liver microsomal cytochrome P-450 isoforms by L-754,394, an investigational human immune deficiency virus protease inhibitor. J Pharmacol Exp Ther 275:1527–1534.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    1. Correia MA
    (1991) Cytochrome P450 turnover. Methods Enzymol 206:315–325.
    OpenUrlPubMed
  7. ↵
    1. Cox SR,
    2. Batts DH,
    3. Ward P,
    4. Freimuth WW,
    5. Staton BA,
    6. Peel BG,
    7. et al.
    (1993) Escalating single and multiple dose safety, tolerance and pharmacokinetics (PK) of delavirdine mesylate (DLV, U-90152S) in normal volunteers [abstract]. National Conference on Human Retroviruses and Related Infections, December 12–16, 1993, p 158.
  8. ↵
    1. Damani LA,
    2. Crooks PA,
    3. Shaker MS,
    4. Caldwell J,
    5. D’Souza J,
    6. Smith RL
    (1982) Species differences in the metabolic C- and N-oxidation and N-methylation of [14C]pyridine in vivo. Xenobiotica 12:527–534.
    OpenUrlPubMed
  9. ↵
    1. Decker CJ,
    2. Rashed MS,
    3. Baillie TA,
    4. Maltby D,
    5. Correia MA
    (1989) Oxidative metabolism of spironolactone: evidence for the involvement of electrophilic thiosteroid species in drug-mediated destruction of rat hepatic cytochrome P450. Biochemistry 28:5128–5136.
    OpenUrlCrossRefPubMed
  10. ↵
    1. Declercq E
    (1994) How to overcome resistance of HIV-1 to HIV-1-specific reverse transcriptase inhibitors. AIDS 8:1020–1021.
    OpenUrlCrossRefPubMed
  11. ↵
    1. Desai D,
    2. Kagan SS,
    3. Amin S,
    4. Carmella SG,
    5. Hecht SS
    (1993) Identification of 4-(methylnitrosamino)-1-[3-(6-hydroxypyridyl)]-1-butanone as a urinary metabolite of 4-(methylnitrosamino)-1-(3-pyridyl)-1-butanone in rodents. Chem Res Toxicol 6:794–799.
    OpenUrlCrossRefPubMed
  12. ↵
    1. Dueweke TJ,
    2. Poppe SM,
    3. Romero DL,
    4. Swaney SM,
    5. So AG,
    6. Downey KM,
    7. Althaus IW,
    8. Reusser F,
    9. Busso M,
    10. Resnick L,
    11. Mayers D,
    12. Lane J,
    13. Aristoff PA,
    14. Thomas RC,
    15. Tarpley WG
    (1993) U-90152, a potent inhibitor of human immunodeficiency virus type 1 replication. Antimicrob Agents Chemother 37:1127–1131.
    OpenUrlAbstract/FREE Full Text
  13. ↵
    1. Gorrod JW,
    2. Aislaitner G
    (1994) The metabolism of alicyclic amines to reactive iminium ion intermediates. Eur J Drug Metab Pharmcokinet 19:209–217.
  14. ↵
    1. Guengerich FP
    (1990) Mechanism-based inactivation of human liver microsomal cytochrome P-450 IIIA4 by gestodene. Chem Res Toxicol 3:363–371.
    OpenUrlCrossRefPubMed
  15. ↵
    1. Kitz R,
    2. Wilson IB
    (1962) Esters of methanesulfonic acid as irreversible inhibitors of acetylcholinesterase. J Biol Chem 237:3245–3249.
    OpenUrlFREE Full Text
  16. ↵
    1. Kronbach T,
    2. Mathys D,
    3. Umeno M,
    4. Gonzalez FJ,
    5. Meyer UA
    (1989) Oxidation of midazolam and triazolam by human liver cytochrome P450IIIA4. Mol Pharmacol 36:89–96.
    OpenUrlAbstract
  17. ↵
    1. Kunze KL,
    2. Trager WF
    (1993) Isoform-selective mechanism-based inhibition of human cytochrome P450 1A2 by furafylline. Chem Res Toxicol 6:649–656.
    OpenUrlCrossRefPubMed
  18. ↵
    1. Laemmli UK
    (1970) Cleavage of structural proteins during the assembly of the head of bacteriophage T4. Nature 227:680–685.
    OpenUrlCrossRefPubMed
  19. ↵
    1. Lopez Garcia MP,
    2. Dansette PM,
    3. Mansuy D
    (1994) Thiophene derivatives as new mechanism-based inhibitors of cytochromes P-450: Inactivation of yeast-expressed human liver cytochrome P-450 2C9 by tienilic acid. Biochemistry 33:166–175.
    OpenUrlCrossRefPubMed
  20. ↵
    1. Lu A,
    2. Levin W
    (1972) Partial purification of cytochrome P-450 and cytochrome P-448 from rat liver microsomes. Biochem Biophys Res Commun 46:1334–1339.
    OpenUrlCrossRefPubMed
  21. ↵
    1. Mani C,
    2. Pearce R,
    3. Parkinson A,
    4. Kupfer D
    (1994) Involvement of cytochrome P4503A in catalysis of tamoxifen activation and covalent binding to rat and human liver microsomes. Carcinogenesis 15:2715–2720.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    1. Masubuchi Y,
    2. Narimatsu S,
    3. Hosokawa S,
    4. Suzuki T
    (1994) Role of the CYP2D subfamily in metabolism-dependent covalent binding of propranolol to liver microsomal protein in rats. Biochem Pharmacol 48:1891–1898.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Miyata M,
    2. Nagata K,
    3. Shimada M,
    4. Yamazoe Y,
    5. Kata R
    (1994) Structure of a gene and cDNA of a major constitutive form of testosterone 6beta-hydroxylase (P450/6beta) encoding CYP3A2: Comparison of the cDNA with P450PCN2. Arch Biochem Biophys 314:351–359.
    OpenUrlCrossRefPubMed
  24. ↵
    1. Omura T,
    2. Sato R
    (1964) The carbon monoxide-binding pigment of liver microsomes: Evidence for its hemeprotein nature. J Biol Chem 239:2379–2385.
    OpenUrlFREE Full Text
  25. ↵
    1. Prough RA,
    2. Ziegler DM
    (1977) The relative participation of liver microsomal amine oxidase and cytochrome P450 in N-demethylase reactions. Arch Biochem Biophys 180:363–373.
    OpenUrlCrossRefPubMed
  26. ↵
    1. Redinbaugh MG,
    2. Turley RB
    (1986) Adaptation of the bicinchoninic acid protein assay for use with microtitre plates and sucrose density gradient fractions. Anal Biochem 153:267–276.
    OpenUrlCrossRefPubMed
  27. ↵
    1. Reinerink EJ,
    2. Doorn L,
    3. Jansen EH,
    4. Van Iersel AA
    (1991) Measurement of enzyme activities of cytochrome P-450 isoenzymes by high-performance liquid chromatographic analysis of products. J Chromatogr 553:233–241.
    OpenUrlCrossRefPubMed
  28. ↵
    1. Rosenberg ZF,
    2. Fauci AS
    (1991) Immunopathogenesis of HIV infection. FASEB J 10:2382–2390.
    OpenUrl
  29. ↵
    1. Sadrieh N,
    2. Thomas PE
    (1994) Characterization of rat cytochrome P450 isozymes involved in the covalent binding of cyclosporin A to microsomal proteins. Toxicol Appl Pharmacol 127:222–232.
    OpenUrlCrossRefPubMed
  30. ↵
    1. Schneck DW,
    2. Pritchard JF
    (1981) The inhibitory effect of propranolol pretreatment on its own metabolism in the rat. J Pharmacol Exp Ther 218:575–581.
    OpenUrlAbstract/FREE Full Text
  31. ↵
    1. Schuetz EG,
    2. Beck WT,
    3. Schuetz JD
    (1996) Modulators and substrates of P-glycoprotein and cytochrome P4503A coordinately up-regulate these proteins in human colon carcinoma cells. Mol Pharmacol 49:311–318.
    OpenUrlAbstract
  32. ↵
    1. Silverman RB
    (1988) Mechanism-Based Enzyme Inactivitation: Chemistry and Enzymology, vol. 1, pp. 3–25. CRC Press, Boca Raton, FL.
  33. ↵
    1. Tassaneeyakul W,
    2. Birkett DJ,
    3. Veronese ME,
    4. Mcmanus ME,
    5. Tukey RH,
    6. Miners JO
    (1994) Direct characterization of the selectivity of furafylline as an inhibitor of human cytochromes P450 1A1 and 1A2. Pharmacogenetics 4:281–284.
    OpenUrlCrossRefPubMed
  34. ↵
    1. vanderHoeven TA,
    2. Coon MJ
    (1974) Preparation and properties of partially purified cytochrome P-450 and reduced nicotinamide adenine dinucleotide phosphate-cytochrome P-450 reductase from rabbit liver microsomes. J Biol Chem 249:6302–6310.
    OpenUrlAbstract/FREE Full Text
  35. ↵
    1. Voorman RL,
    2. Maio SM,
    3. Hauer MJ,
    4. Sanders PE,
    5. Payne NA,
    6. Ackland MJ
    (1998) Metabolism of delavirdine, an HIV-1 reverse transcriptase inhibitor, by microsomal cytochrome P450 in humans, rats and other species: Probable involvement of CYP3A subfamily. Drug Metab Dispos 26:631–639.
    OpenUrlAbstract/FREE Full Text
  36. ↵
    1. Weber C,
    2. Stoeckel K,
    3. Lalka D
    (1994) Accumulation kinetics of propranolol in the rat: comparison of the Michaelis-Menten-mediated clearance and clearance changes consistent with the altered enzyme hypothesis. Pharmaceut Res 11:420–425.
    OpenUrlCrossRefPubMed
  37. ↵
    1. Woolf TF,
    2. Pool WF,
    3. Bjorge SM,
    4. Chang T,
    5. Goel OP,
    6. Purchase CF 2d,
    7. Schroeder MC,
    8. Kunze KL,
    9. Trager WF
    (1993) Bioactivation and irreversible binding of the cognition activator tacrine using human and rat liver microsomal preparations. Drug Metab Dispos 21:874–882.
    OpenUrlAbstract
Back to top

In this issue

Download PDF
Article Alerts
Email Article
Citation Tools

Research ArticleArticle

Microsomal Metabolism of Delavirdine: Evidence for Mechanism-Based Inactivation of Human Cytochrome P450 3A

Richard L. Voorman, Stephen M. Maio, N. Ann Payne, Zhiyang Zhao, Kenneth A. Koeplinger and Xiaohong Wang
Journal of Pharmacology and Experimental Therapeutics October 1, 1998, 287 (1) 381-388;
Twitter logo Facebook logo Mendeley logo

Jump to section

Advertisement