The word "China" means two different things, related but distinct:
Definition 1. A large country in East Asia. It was founded in 1949 and the capital is Beijing. In this sense, 'China' is short for 'People's Republic of China'. You can also use this word for now-defunct countries. In 1900, for example, 'China' referred to a large country in East Asia which was ruled by Manchus from their capital in Beijing (or Peking).
Definition 2. A civilization centered in East Asia. Despite a high level of diversity and large regional differences, it is considered a fairly coherent entity. It is distinct from Korean, Japanese, and Vietnamese culture, although it has influenced them a great deal. It is much, much older than the People's Republic of China.
The two definitions above are distinct. I propose that if you use the word 'China' (or 'Chinese') in a sentence, it should be easy to discern whether you mean definition 1 (the country) or definition 2 (the civilization). If it's a weird amalgamation of the two, there is an excellent probability that you are using language to obfuscate and muddle. Shame on you.
You might argue that one could say something similar for every country in the world. (France the entity headed by Nicholas Sarkozy vs. France the culture, and so on.) You would be right.
But China is a particularly important case. As Martin Jacques said in the one bit of wisdom in his TED talk otherwise filled with dubious assertions, China isn't a nation-state; it's a civilization-state. It can be thought of as something akin to Europe, if we had an odd parallel Europe where the centuries-old political trend was toward unified empires, and there was a single language so dominant that all the other languages were reduced in people's minds to the status of 'dialects'.
But that's definition 2 above. There's also definition 1, where 'China' refers to a country that's just one country in the world out of 200. There are plenty of ways the two definitions do not overlap perfectly, and as a result you see both innocently sloppy thinking and deliberate obfuscation based on the fact we've got one word with two meanings.
For example, take the idea that China has 5000 years of history, while American history started in 1776. It sure sounds like China's got a long head start on the USA from those numbers, but they're derived entirely from the obfuscatory power of muddled language. If we're talking about countries (definition 1), the PRC was only founded in 1949, so the USA is actually a great deal older. If we're talking about civilizations (definition 2), there was no civilization that started in 1776. It's not as if Thomas Jefferson was some sort of legendary culture hero who invented the Roman alphabet as he scratched out the words We hold these truths to be self-evident which was the earliest recorded Western writing. 
But that's just sloppy thinking, really. People use this sort of semantic vagueness to deliberately muddle issues as well. Let's think about the distinction between definition 1 and 2 in this sentence: 'Taiwan is Chinese'.
If we go by definition 1, that means 'Taiwan is a part of the People's Republic of China'. Which, despite what Beijing would wish, is obviously not true.
But if we go by definition 2, we get 'Taiwan is a part of Chinese civilization'. You could argue about that, but simply on its own merits I don't have a huge objection to it. Taiwan has plenty of native-born Taiwanese who consider themselves Chinese. (And the vast majority would be horrified if Taiwan suddenly became part of the PRC tomorrow. They don't think China = PRC.) 
If you think of China as a civilization akin to Europe, you'd have to agree that if the EU was a country and (say) Ireland was not part of the EU, it would still be culturally European.
But we live in a universe where people who wish Taiwan was ruled by Beijing have no qualms about telling Westerners 'Of course Taiwanese are Chinese! Taiwan is a province of China and the Taiwanese people identify as Chinese!' and thinking up terms like 'Chinese Taipei'. They are well aware that when the vast majority of human beings hear 'China' or 'Chinese', they think of that large country in East Asia, whose capital is Beijing.
In other words, they're playing with language, preying on the fact that to most Westerners 'China' means only one thing: "The People's Republic of China". I don't like it. It grates uncomfortably against my brain. I want people to stop doing it.
That's the context for why, until advocates for Taiwan's absorption into the PRC stop playing with words, I will never be entirely comfortable hearing about how Taiwan's culture is Chinese.
 Which doesn't even get into the fact that you only get 5,000 years of Chinese history if you count about 1,800 years of tradition and legends that predate the earliest surviving Chinese written records, which stretches the word 'history' rather far for my comfort.
 Even so 'Taiwan is culturally Chinese' is still an imperfect assertion, as Taiwan's got aborigines who are not culturally Chinese.