Is it primarily the skill building required or is it also the sense of team it builds in the recruits since they have to go through "hell" together, or is it something else entirely?
I'll speak about the Army's BCT program, here. I slightly disagree with Jon Davis on the assertion that the Marine boot camp is the "hardest". Instead I'll posit that each entrance program into the military branch it is designed for is "hard" in areas where said branch needs assurances that the soldier it trains is ready to perform the job they are being trained and paid for. A lot of things, not the least of which is actual human lives, depend on a soldier's readiness. And this extends far beyond combat and far beyond conflict type situations.
Army BCT (Basic Combat Training) is designed to slowly raise demands over its course and not only prepare the soldier for AIT, Advanced Individual Training,but also to weed out those who can not properly perform the functions required for a job in the field. In this regard BCT in the Army is not unlike MFT, Medical Field Training, for EMT or FT, Fire Training, for Firefighters. It's pretty similar to Cop Academy, too.
There is no "hell" in BCT. BCT ascertains some things that might be foreign or new to the average 18 year old soldier in training and considered less important to civilians performing civilian jobs, but nothing in BCT is "hell". The shouting you see in movies and documentaries or you hear from soldiers about is a carefully crafted tool to work on a human instinct - to associate the sound with the message. In Jonas Mikka Luster's answer to Do drill sergeants yell as much as they are depicted to yell? What is the purpose of this shouting at each other? I explain this, how loud conversation is important because the message is never just for one recruit and because soldiers learn to strip the sound from the message and go from "shit, that's loud and scary" to "what does this mean?".
Other than that a number of very basic abilities are tested and trained. The ability to sleep and wake at specified times. The ability to hear, understand, and follow orders by a superior. Physical fitness to perform tasks, and mental and emotional fitness to do so without breaking. Contrary to popular belief the Army does not want robots or zombies, the Army wants intelligent people who use said intelligence to ask "how best" instead of "why". For the "why" we have computers and planners and manuals and politicians. For the "how best" we need soldiers who think on their feet.
Any soldier leaving BCT has been vetted by their training unit to have the physical, mental, technical, and instructional tools to be trained at very high cost to the tax payer and the Army for their job at hand. In AIT those assessments are continued, an Army cook like me is evaluated over different needs than an Infantryman or military cop. But in the end of ends when the Army needs cops and less cooks and drags someone from the kitchen back into the classroom they know that, way back in BCT, he or she acquired all the tools and knowledge and was checked and evaluated for their ability to perform that job as well.