In order to understand the great tumult that is occuring within the Torah world today, it is important to review the general philosophies of the major players and see how their approaches clash with one another.
Currently, the dominant approach is that of the Chareidi community. Based on the famous teaching of the Chasam Sofer, zt"l, that "everything new is forbidden by the Torah", this community bases its religious philosophy on opposition to any innovation by Jewish law to accomodate social, political and cultural change. Why? The name of the community itself gives us a vital clue. Chared means to tremble and the Chareidim believe that they tremble before God. Their overriding desire is to spend life avoiding doing anything which is contrary to the will of the All-Mighty and to perform those duties they believe they were created to do in order to justify God's reason for creating them in the first place.
With this explanation, one can understand better the reason for strictness that is so prevalent in the Chareidi mindset, especially nowadays. To foribd something that might be questionable is meritorious because it fulfills the goal of avoiding offending God or not fulfilling all His expectations of us. Spending time learning Torah, which we know from our holy sources is the Jew's highest duty is the best way to live a life that has meaning, that is the Jewish ideal.
The negative side to this is that the Chareidi mind does not think laterally. A Chareidi scholar may have thousands, or tens of thousands of pages of material memorized, may show analytical skills that outshine the greatest secular thinkers in the world, but he will generally confine his knowledge base to a small corner of the world, to those sources which best represent his hashkafah and worldview. Outside knowledge, the opinions of educated religious scholars who don't share the Chareidi opinion on things, are generally ignored or scorned. In other words, however complex and deep the Chareidi viewpoint on something, it is at the same time shallow for it will not confront that which contradicts it.
In contrast, the Modern Orthodox philosophy is far more open. Based on the examples of Rav Shimshon Rafael Hirsch, Rav Azriel Hildesheimer and the Rav J.B. Soloveitchik, Modern Orthodox belief holds that the entire universe, all knowledge in existence, is part of God's creation and therefore an adjunct to learning Torah. If science, archeology or history contradict that which the Torah tells us, then the challenge for the Modern Orthodox scholar is to reconcile the two views so there is harmony between them and the truth of Torah remains intact, as it must.
The downside of this approach is the exact opposite of its Chareidi counterpart. Instead of ignoring and avoiding secular knowledge, the Modern Orthodox mind embraces it but at the same time can be seduced away from Torah learning by it. One can easily slip from the position where secular knowledge is an adjunct to Torah to a more precarious and questionable point where it develops a value of its own, independent from Torah. In many parts of the Modern Orthodox world, this has already happened, especially on the "left" side of the community where many have raised secular values that are in and of themselves antithetical to Torah to have an overriding importance of their own. If the Chareidi scholar develops a philosophy that can only survive in a ghetto, the Modern Orthodox scholar can creat a worldview in which Torah is only one option amongst many to be chosen, an even worse position for a believing Jew to be in.
Ultimately, there must be a compromise between these two positions for the Jew who wishes to be fervent in his belief in the truth of God's Torah but who wishes to know that the Torah is in consonance with the rest of the universe, especially as Chazal tell us it is the blueprint of that universe.
Thus a middle position would be one in which the primacy of Torah is always upheld. The Rav, in Ish Halachah, notes a profound observation about the dual nature of man. One part he calls "cognitive man". Cognitive man wishes to understand how reality works. He develops laws, rules and regulations. The universe must perform in an orderly, unalterable fashion for him. The contrasting facet is homo religiosis, the man of faith for whom the mysteries of the universe are paramount. There are some things that a human being, with all his limitations and frailties, cannot understand and the more a person has an awareness of this, the more he is in awe of the majesty and power of his Creator.
For the Rav, it is the reconciliation between these two characteristics that leads to the ideal Jew. An simple example will suffice. When a person first studies cells in biology, the concept seems relatively simple. There are two types of cells: animal and plant. Cells make up all living creatures. They eat and excrete.
But then comes further data on the components of cells and then the components of the components. After that, organic chemistry and an understanding of the various metabolic cycles that maintain the cell's functions. And let us not forget DNA, RNA, protein transcription, chemical bones, atomic structure, and so on. The more one learns about the biology of the cell, the more one realizes how huge and complex this microscopic unit is and how much there still is about it that we do not yet understand. The more we learn, the less we know.
This is, according to the Rav, the proper synthesis of cognitive man and homo religiosis. The former struggles to learn everything about everything. The latter realizes that the more he learns, the less he understands until he finally stands in true awe and appreciation of the complexity of life and God above it.
Therefore, the purpose of a person in this world is not to simply sit and study Torah. If one does, one learns much about the mysteries of Judaism and God but misses out on the wonders of God's creation. One never strives to place everything in order and as a result, never develops a depth of awe that would be expected of him. And the mishnah in Avos confirms this: "Study is not the main thing, doing is."
Yet the opposite, spending one's life learning about the external world and not learning Torah also leads in the wrong direction. Of what use is a degree in Physics if it does not lead the holder to a greater appreciation of God's desires and demands of us? Such development of understand is the point of all learning for the only meaningful paths of knowledge lead to a greater understanding of Torah and God's will. Thus again the mishnah in Avos tells us that "everything is in Torah" because even those sources of knowledge outside the religious sphere must be learned and understood within Torah's holy framework.
The final synthesis then must be the idea that learning of Torah adjuncted by understanding of the outside world will lead to a more complete appreciation of God and a better life led by us. May we all merit to walk in the Torah's ways of pleasantness.