An article on this same topic appeared some time ago in the Forward in Christ magazine. It is copied below and should offer an adequate answer:

Question: Who were the "sons of God," "daughters of men," and "the Nephilim" mentioned in Genesis 6:1-4? And how did the Nephilim survive the Flood?

Answer: These verses have generated more than their share of imaginative interpretations and have become a breeding ground for fantastic speculation. Thanks for allowing us to review options and offer comments.

More Bizarre but Less Likely Option

The less plausible yet persistently popular idea is that Genesis 6 is talking about fallen angels ("sons of God") who impregnated select beautiful women ("daughters of men") and the demon-human offspring ("the Nephilim") were giants in stature and accomplishment prior to the Flood. Since Nephilim are spoken of after the Flood (Numbers 13:33), this race of giants somehow survived the Flood (or was restarted by another, post-Flood, demonic invasion with sexual unions with humans).

This scenario, of course, is exciting stuff that grabs people's attention. Superhuman giants! Hybrid creatures! And, stated honestly, it is grammatically and linguistically possible to arrive at this conclusion through a reading of Genesis 6:1-4. To say this view is utterly impossible or flagrantly contrary to Scripture is perhaps an overstatement.

Less Bizarre and More Likely Option

The less exotic understanding of this section, but one more compatible with everything else revealed in the Bible, is this: Male descendents of Adam and Eve through Seth (the dominant line of believers, "sons of God") intermarried with attractive female descendents of Cain (the dominant line of unbelievers, "daughters of men") resulting not only in a deterioration of religious principle, but also in aggressive children who became strong in activity and reputation ("the Nephilim").

The term Nephilim is most likely derived from the Hebrew nphl ("to fall" or "fall upon") and refers to "fallen" people (unbelieving rebels against God) or aggressive bullies who "fall on" others (overpowering and tyrannizing them). Nephilim might also derive from the root pl' ("to be awesome, full of wonder"), and the title then stresses they were people who were strong in physical stature, accomplishment, and reputation – including (if ancient traditions are considered) bullying others as gangsters or mobsters.

These early Nephilim perished in the Flood (Genesis 7:21), but other giants in stature developed in later generations and family branches of mankind. The term Nephilim need not refer to a specific race or tribe, but to people who bore the same general characteristics. Included among them were the Anakim, Rephaim, and Emim mentioned in Numbers 13:33, Deuteronomy 3:11, and 2:10. The Philistine warrior Goliath is probably the best known example of an aggressive giant (1 Samuel 17:4), but there is evidence that people of exceptional size lived in various parts of the world through most of history.

The Preferable Option

One could argue that both options outlined above are possible and the relatively obscure references in Genesis 6 should lead us to advocate no preference between them. One reason we list the second scenario as more plausible is that the idea of angels being capable of, interested in, or allowed by God to impregnate humans is simply foreign to the rest of Scripture. The words of Jesus in Mark 12:24-25 lead us to conclude that angels are not sexual beings the same way humans are. Notions of angel/human hybrids stem from later, non-biblical sources.

There's a second reason we may prefer the view about believers compromising religious principle through bad choices (including selection of spouses) that often results in ethically challenged progeny. This reason is more theological than textual or exegetical. It fits the pattern so often warned against yet so often repeated in subsequent generations of mankind. This way of deriving the meaning of a text is not adequate in and of itself – but when the conclusion is fully compatible with everything else the Bible says on a given subject, it may be seen as preferable.