Drug development is a lengthy process, taking approximately 10-15 years. The process, according to the US Food & Drug Administration (FDA), includes 5 stages: Drug Discovery, Preclinical research, Clinical research, FDA review, and FDA post-market monitoring.
Ideally, the process starts with identifying a "target" related to the disease studied. Chemicals that affect the "target" are identified and modified to design the drug. However, this technique has just been recently made possible with modern scientific and technological advancements. So, how were drugs discovered beforehand?
In fact, many of the most used drugs today have unknown mechanisms, such as psychotropic drugs (lithium for schizophrenia), paracetamol, one of the most known pain relievers, and even penciling. In order to understand why we don't know how these drugs work, we have to first learn how they were discovered.
Acetaminophen, widely known as paracetamol, was discovered by accident in 1893 when Joseph Mering reacted p-nitrophenol with tin and glacial acetic acid. There was some competitiveness between paracetamol and phenacetine, but after many years, paracetamol proved to have fewer side effects. Today, research is being conducted to understand its analgesic mechanism, that is, in the hope of creating better drugs that have a similar mechanism, but even fewer adversities. The reason why we still do not have a satisfactory understanding of how this drug works is because paracetamol most likely has a pharmacological mechanism that interacts with many physiological pathways.
Another drug that was born by accident is penicillin. As Dr.Macfarlane describes its discovery: "A series of chance events of almost unbelievable improbability". After coming to his lab again from vacation, Alexander Flemming noticed a zone around an invading fungus on an agar plate in which the bacteria didn't grow. He then isolated the mold and extracted the active agent from it, naming it "penicillin". However, since much of the focus was on how to bring the drug to the market through WW1 and WW2, the mechanism remained a mystery for 80 years, before an international team of researchers led by the University of Sheffield discovered that β-lactam antibiotics kill MRSA (Methicillin Resistant S. aureus) by creating holes in the cell wall which enlarge as the cell grows, eventually killing the bacteria.
Even though drugs are now methodologically developed, many discoveries were the result of accidents, or through a combination of trial and error experimentation and observation of human and animal reactions as a result of ingesting such products. In the modern day, scientists are working to understand the mechanism with which those drugs work, but face a lot of struggle because of the difficulty of getting funding, or maybe the drug's effect on multiple physiological pathways, and thus, the discovery of new drugs with similar mechanisms will have to be embedded until this narrative changes.
References:
Toussaint, K., Yang, X. C., Zieliński, M., Reigle, K. L., Sacavage, S., Nagar, S., & Raffa, R. B. (2010). What do we (not) know about how paracetamol (acetaminophen) works? Journal of Clinical Pharmacy and Therapeutics, 35(6), 617–638. https://doi.org/10.1111/j.1365-2710.2009.01143.x
Gaynes, R. P. (2017). The Discovery of Penicillin—New insights after more than 75 years of clinical use. Emerging Infectious Diseases, 23(5), 849–853. https://doi.org/10.3201/eid2305.161556
Scientists make breakthrough in understanding how penicillin works. (2021, October 26). The University of Sheffield. https://www.sheffield.ac.uk/news/scientists-make-breakthrough-understanding-how-penicillin-works
Office of the Commissioner. (2018). The drug development process. U.S. Food And Drug Administration. https://www.fda.gov/patients/learn-about-drug-and-device-approvals/drug-development-process
Cancer Research UK. (2022, March 3). How long a new drug takes to go through clinical trials. https://www.cancerresearchuk.org/about-cancer/find-a-clinical-trial/how-clinical-trials-are-planned-and-organised/how-long-it-takes-for-a-new-drug-to-go-through-clinical-trials#:~:text=There%20is%20no%20typical%20length,a%20drug%20to%20be%20licensed.
Drug Discovery, & Further Reading. (2008). Appendix 1: History of drug discovery and development. In Drugs (pp. 391–397). Wiley. https://doi.org/10.1002/9780470403587.app1