The obvious answer is, of course, that silk is derived from silkworms, and worms are animals. Using them in the production of fabric at all goes against the anti-slavery ethic of animal rights. But it's also about what they need to do to the silkworms in order to produce the silk.
First, the cocoons with silkworms inside of them are brought into the factory. The cocoons might come from a farmer, or a scientific lab. The coccons are then sorted according to colour, shape, etc., since these factors influence the "quality" of the silk once it has been made.
The silkworms and their cocoons are boiled alive to extract the thin strands of silk. (To produce a metre-long cloth, 1500 worms must be killed.) The silk is further processed to make it into the "silky" finished product.
Boiling silkworms alive is definitely not animal-friendly. Gandhi advocated against this form of silk production, and PETA is anti-silk as well.
Some people talk about "Ahisma silk", or "Peace Silk". "Ahisma" comes from the work for "non-violence" in Hindi. Silkworms raised for Ahisma silk are let to stay in their cocoons until they turn into moths, at which point they fly out of their cocoons. Once the cocoons are empty, the process of extracting the silk begins. However... Ahisma silk isn't really non-violent, according to Indian Vegan (http://www.indianvegan.com/articles/Compassionate%20Friend%20Monsoon%202009%20Vol%20XXXII%20No%202.pdf). The moths used for Ahisma silk are used to mate and kept in barbaric conditions (stuffed in freezers) and then destroyed. They come out of their cocoons deformed because of the process their cocoons have been put through. Therefore, not even "Peace Silk" is truly vegan. An excellent website on why "Ahisma Silk" is not good can be found here: http://www.wormspit.com/peacesilk.htm.
There aren't many sources available on silk as it pertains to veganism, but here are one or two sources of information that you might find useful:
There are alternatives to silk, of course, that you can use instead. You can get ties, for example, made of polyester and/or cotton. There are wonderful silky materials that you can use in place of silk, such as rayon, which is 100% vegan. Other alternatives include "nylon, polyester, Tencel, milkweed seed-pod fibers, [and] silk-cotton tree filaments", according to someone on Yahoo! Answers.
Have a great valentine's day! Make sure to give your valentine some vegan chocolate! (Endangered Species brand is a good one for dark chocolate, but there are plenty of more fancy vegan chocolate brands out there, too.)