CS 259D 僵尸网络 BotFinder: Finding Bots in Network Traffic Without Deep Packet Inspection

作者:Leo

  • 背景知识与启发
  • 论文目标与贡献
  • 数据来源
  • BotFinder
    • 符号定义
    • 输入数据处理
    • 流量重组
    • Trace 提取
    • 统计特征分析
    • 模型创建(训练)
    • 恶意软件检测
  • 僵尸机器人演进
    • 僵尸网络的策略
    • 僵尸网络的成本
    • BotFinder 检测失败的可能原因
  • 参考资料

背景知识与启发

  • 三个核心设计目标
    • 能够检测单个机器人感染
    • 只依靠高层次/网络流量提供的信息
      • 不检查载荷
      • 抵御加密通信
      • 数据信息容易获得
    • 对静默的机器人也适用
      • 窃取敏感数据
      • 没有“嘈杂”的行为
        • 垃圾邮件
        • DoS
  • 观测到的现象
    • C&C 连接遵循一定的模式
    • 僵尸机器人向 C&C 发送相似的数据
    • 以相似的方式将信息上传到 C&C
    • 与 C&C 的通信时间模式有规律可循
  • 在可控的环境中运行僵尸程序,即可学习到僵尸程序的活动模式

论文目标与贡献

  • 观测不同僵尸网络家族的 C&C 流量呈现出的规律
  • 使用机器学习的方法,自动生成僵尸检测模型
  • BotFinder 原型机
    • 一个通过监视网络流量,来检测单个受感染宿主的系统

数据来源

  • 2011年6月的30天之内,在 Anubis 上采集到的活跃的恶意软件样本
  • 每个僵尸家族平均32个样本

BotFinder

BotFinder 通用架构

符号定义

  • \(\tau\), trace ,两个网络端点间数据流动的时间序列
  • \(M\),模型
  • \(\tau_M\), 模型 \(M\) , trace \(\tau\), 的分数
  • \(q_{cluster}\), 质量评级

输入数据处理

  • 所有网络流量
  • 扫描流量,通过:
    • VirusTotal
    • Anubis
  • 容忍杂讯

流量重组

  • 依照 NetFlow 汇总数据

Trace 提取

不同统计模式的 trace

  • \(|\tau|_{min}\),最少通信次数 (经验值 10 到 50)
  • 宿主与 C&C 服务器之间的多次通信组成了命令与控制通信
  • 通过两种方法,过滤流量,并识别相关的流量 trace
    • 白名单
    • 第三方信息

统计特征分析

  • 一个 trace 中两个子序列的启动时间的平均时间间隔
    • Botmaster 必须确保所有的机器人在他的控制之下
    • 通信不遵循 push 模式
      • 宿主可能位于 NAT 内
      • 未在 C&C 注册
    • 通信有固定的时间间隔,并且呈现出宽松的周期性
  • 平均通信持续时间
    • 期望持续时间是相似的
  • 传送到源的平均字节数
  • 传送到目的的平均字节数
  • 对数据流起始时间做快速傅立叶变换(FFT)
    • 确定通信的基本频率

模型创建(训练)

  • 为每个特征分别聚类
    • 恶意软件的特征是不相关的
  • 丢弃小且多样性高的集合(聚类评级较低)
  • CLUES 算法
    • 允许无参数聚类
    • 比 K-means 更好
  • 聚类质量评级
    • 大且数值相近的集群更好
    • \(q_{cluster} = exp(-\beta \frac{SD}{Avg})\)
      • \(\beta = 2.5\) (经验值)

恶意软件检测

  • 将 trace 的每个特征与相应模型的集群进行匹配
  • \(\tau\) 命中上了 \(M\) 的一个特征
  • 把 \(q_{cluster} \cdot exp(-\beta \frac{SD_{trace}}{Avg_{trace}})\) 加到 \(\tau_M\)
    • \(\beta = 2.5\) (经验值)
  • 为每个模型维护一个 \(\tau_M\)
  • 把最高的 \(\tau_M\) 与阈值 \(a\) 进行比较
  • 允许设定命中特征数量的最小值, \(h\)

僵尸机器人演进

僵尸网络的策略

  • 行为更加随机
    • 当僵尸网络的行为 100% 随机时,BotFinder 的检测率保持在 60%
  • 时间间隔更长
    • 快速傅立叶变换能够克服
  • C&C 服务器频繁变化
    • BotFinder 无法构建 \(|\tau|_{min} = 50\) 的 trace
      • 没有观测到如此高的 C&C 服务器变化 (IP flux)
    • 可以在步骤 4 之前添加一步预处理
      • 合并两个子 traces :\(\tau_A\) 和 \(\tau_B\)
      • 两个影响因素
        • \(\tau_{AB}\)的标准差要比二者中最小的还小
        • \(\tau_{AB}\) 的 \(q_{cluster}\) 要高于阈值
  • P2P 僵尸网络
  • 僵尸网路模拟正常通信

僵尸网络的成本

  • C&C 和宿主需要更新僵尸网络通信拓扑信息
  • 增加僵尸网络运营商的成本,降低僵尸网络的稳定性和性能

BotFinder 检测失败的可能原因

  • 僵尸网络通信明显随机化
  • 大幅增加通信间隔,导致 BotFinder 必须捕捉长时间的 trace
  • 通信源和目的加入多变的的通信开销
  • 非常频繁地更换 C&C 服务器
  • 每次 C&C 服务器更改后使用完全不同的通信模式

参考资料

  • BotFinder: Finding Bots in Network Traffic Without Deep Packet Inspection
  • CS 259D Lecture 2

“CS 259D 僵尸网络 BotFinder: Finding Bots in Network Traffic Without Deep Packet Inspection”的117个回复

  1. I simply want to tell you that I am very new to blogging and site-building and really loved your blog site. Almost certainly I’m going to bookmark your website . You surely come with amazing article content. Thanks a bunch for revealing your web site.

  2. It іs appropriate time to make some pⅼans for thefuture and it’s time to be hаpρy. I’ve read this post andif I coulԁ I ᴡish to suggest you some interdesting things or suggestions.Perhaps you can write next articles referring to thisarticle. I ɗesire to read more things abߋut it!

  3. This is very fascinating, You are an overly skilled blogger.I’ve joined your rss feed and look forward to in quest of extra of your fantasticpost. Additionally, I have shared your site in my social networks

  4. Its like you read my mind! You appear to know so much about this, like you wrote the book in it or something. I think that you could do with a few pics to drive the message home a little bit, but other than that, this is great blog. A fantastic read. I will certainly be back.

  5. hi!,I really like your writing very a lot! proportion we keep up a correspondence more about your article on AOL? I require a specialist on this area to solve my problem. May be that is you! Taking a look ahead to look you.

  6. I do not even know how I ended up here, but I thought this post was great. I do not know who you are but certainly you’re going to a famous blogger if you aren’t already 😉 Cheers!

  7. I like the valuable information you provide in your articles. I will bookmark your weblog and check again here frequently. I’m quite certain I’ll learn a lot of new stuff right here! Good luck for the next!

  8. Emery EPS is a Search Engine Optimization company that provides SEO Services. Their proprietary SEO strategies help struggling websites and aspiring business owners to rank their websites higher in multiple search engines like Google , Yahoo and Bing. They provide local and gmb map ranking for businesses in Coasta Rica, New York, Los Angeles, Portland, Sacramento, Phoenix, Las Vegas, Denver, Houston, California and many other local areas. Find more at https://www.emeryeps.com.

  9. RETSPRO ripoffs have been around on the internet, so please beware of trying to buy versions of our software at reduced prices. There is only one legitimate place to buy our software and that is on our own RETS PRO website.

  10. I wish to show my admiration for your kind-heartedness for men and women that require guidance on this important area of interest. Your personal commitment to passing the solution up and down appears to be wonderfully practical and have really made men and women like me to get to their pursuits. The warm and friendly useful information denotes a lot a person like me and especially to my peers. Thank you; from each one of us.

  11. Thank you for the sensible critique. Me and my neighbor were just preparing to do some research about this. We got a grab a book from our local library but I think I learned more from this post. I’m very glad to see such excellent info being shared freely out there.

  12. Thanks, I have just been searching for information about this topic for a long time and yours is the greatest I’ve discovered till now. However, what about the bottom line? Are you positive about the supply?

  13. Pretty nice post. I just stumbled upon your weblog and wanted to say that I’ve truly enjoyed browsing your blog posts. In any case I’ll be subscribing to your rss feed and I hope you write again very soon!

  14. I’ve been browsing online more than three hours nowadays, yet I never found any attention-grabbing article like yours. It¡¦s lovely value enough for me. In my view, if all site owners and bloggers made just right content as you did, the web will likely be a lot more useful than ever before.

  15. I¡¦m not positive where you’re getting your information, but great topic. I must spend some time studying more or figuring out more. Thanks for great information I used to be in search of this info for my mission.

  16. I’m just commenting to make you understand of the great discovery my child undergone going through your site. She came to find a wide variety of pieces, which include how it is like to possess an incredible teaching mindset to let a number of people quite simply comprehend various hard to do topics. You actually exceeded our expectations. Thank you for imparting these priceless, safe, informative and also easy tips about your topic to Jane.

  17. hey there and thank you for your information – I have definitely picked up something new from right here. I did however expertise several technical points using this site, since I experienced to reload the web site lots of times previous to I could get it to load correctly. I had been wondering if your web hosting is OK? Not that I’m complaining, but slow loading instances times will sometimes affect your placement in google and could damage your high-quality score if advertising and marketing with Adwords. Anyway I am adding this RSS to my email and can look out for much more of your respective intriguing content. Ensure that you update this again very soon..

  18. My wife and i have been really ecstatic that Ervin could round up his web research with the ideas he received through the site. It is now and again perplexing to simply choose to be giving freely information some others have been making money from. And now we do know we have got the website owner to thank because of that. All the explanations you’ve made, the simple web site navigation, the relationships you can help instill – it is everything sensational, and it is helping our son in addition to our family reason why the idea is entertaining, which is seriously indispensable. Thanks for all the pieces!

  19. Thank you for the sensible critique. Me and my neighbor were just preparing to do a little research about this. We got a grab a book from our local library but I think I learned more clear from this post. I’m very glad to see such excellent info being shared freely out there.

  20. I want to show some appreciation to the writer for rescuing me from this problem. Right after looking out throughout the the web and finding strategies that were not beneficial, I thought my life was done. Existing without the presence of solutions to the issues you have fixed as a result of your review is a crucial case, as well as the ones which may have badly damaged my entire career if I hadn’t discovered the website. Your own personal understanding and kindness in handling all the pieces was precious. I’m not sure what I would’ve done if I had not encountered such a solution like this. I can now relish my future. Thanks so much for the expert and amazing help. I will not be reluctant to propose your web site to any individual who requires tips about this subject.

  21. I want to show my admiration for your kindness for all those that absolutely need help on the concept. Your special dedication to getting the solution all over was exceedingly good and has continually enabled others like me to get to their ambitions. This warm and friendly advice entails a whole lot a person like me and somewhat more to my fellow workers. Thanks a ton; from all of us.

  22. Great – I should certainly pronounce, impressed with your site. I had no trouble navigating through all tabs and related information ended up being truly simple to do to access. I recently found what I hoped for before you know it in the least. Reasonably unusual. Is likely to appreciate it for those who add forums or anything, website theme . a tones way for your customer to communicate. Nice task.

  23. I must show some appreciation to the writer just for rescuing me from this trouble. Because of looking out throughout the the net and getting principles which are not productive, I thought my entire life was well over. Being alive without the approaches to the problems you have fixed by means of your site is a crucial case, and those which might have in a negative way affected my entire career if I had not encountered your web page. The understanding and kindness in dealing with the whole thing was important. I am not sure what I would have done if I had not discovered such a thing like this. I am able to at this point look ahead to my future. Thank you very much for this skilled and amazing help. I won’t be reluctant to refer your web blog to anyone who wants and needs support on this problem.

  24. Hey very nice blog!! Man .. Excellent .. Wonderful .. I’ll bookmark your blog and take the feeds also¡KI am happy to search out so many useful info right here in the put up, we need develop more strategies in this regard, thank you for sharing. . . . . .

  25. Thanks for some other informative blog. Where else may just I get that type of information written in such a perfect means? I have a project that I’m just now running on, and I have been at the glance out for such information.

  26. I alwaysconstantlyevery time spent my half an hour to read this blogweblogwebpagewebsiteweb site’s articlespostsarticles or reviewscontent everydaydailyevery dayall the time along with a cupmug of coffee.

  27. Simply desire to say your article is as astounding. The clearness in your post is just nice and i could assume you’re an expert on this subject. Well with your permission let me to grab your RSS feed to keep updated with forthcoming post. Thanks a million and please carry on the rewarding work.

  28. Hiya very cool website!! Guy .. Excellent .. Superb .. I will bookmark your web site and take the feeds also¡KI am happy to search out numerous useful information here in the put up, we want develop extra techniques in this regard, thanks for sharing. . . . . .

  29. I would like to thnkx for the efforts you’ve put in writing this site. I’m hoping the same high-grade website post from you in the upcoming as well. Actually your creative writing abilities has inspired me to get my own web site now. Actually the blogging is spreading its wings rapidly. Your write up is a great example of it.

  30. What i do not realize is if truth be told how you are now not actually a lot more well-preferred than you might be right now. You’re very intelligent. You understand therefore considerably in the case of this matter, produced me in my opinion consider it from numerous varied angles. Its like women and men don’t seem to be interested unless it¡¦s one thing to do with Lady gaga! Your personal stuffs nice. At all times handle it up!

  31. You could certainly see your skills in the paintings you write. The arena hopes for more passionate writers like you who aren’t afraid to mention how they believe. Always follow your heart. “The only way most people recognize their limits is by trespassing on them.” by Tom Morris.

  32. I really like your writing style, good information, appreciate it for posting :D. “I hate mankind, for I think myself one of the best of them, and I know how bad I am.” by Joseph Baretti.

  33. I must show thanks to the writer for rescuing me from this particular condition. As a result of browsing throughout the internet and getting concepts which are not powerful, I believed my life was gone. Existing without the strategies to the difficulties you have sorted out by means of this website is a crucial case, and those which may have negatively damaged my career if I had not noticed your website. Your own personal skills and kindness in dealing with a lot of stuff was helpful. I am not sure what I would’ve done if I hadn’t come upon such a step like this. I can at this moment relish my future. Thanks for your time so much for this expert and sensible help. I will not hesitate to suggest the website to anybody who needs and wants guidelines on this topic.

  34. Thank you a lot for giving everyone remarkably brilliant possiblity to read critical reviews from this website. It really is so awesome and jam-packed with fun for me and my office mates to search your web site minimum thrice in 7 days to read through the fresh issues you have got. And indeed, I’m so actually amazed concerning the perfect tips and hints you serve. Some 2 points in this post are honestly the most beneficial I have had.

  35. We are a group of volunteers and starting a new scheme in our community. Your website provided us with valuable info to work on. You’ve done an impressive job and our entire community will be thankful to you.

  36. Good ¡V I should certainly pronounce, impressed with your website. I had no trouble navigating through all tabs as well as related information ended up being truly easy to do to access. I recently found what I hoped for before you know it in the least. Reasonably unusual. Is likely to appreciate it for those who add forums or something, web site theme . a tones way for your customer to communicate. Excellent task..

  37. Merely wanna remark on few general things, The website style is perfect, the content is real great. “The reason there are two senators for each state is so that one can be the designated driver.” by Jay Leno.

  38. I together with my guys came reading through the great recommendations from the blog then all of a sudden I got a terrible feeling I had not expressed respect to the web blog owner for those tips. Those men appeared to be totally passionate to learn all of them and already have actually been taking pleasure in these things. Appreciation for truly being well helpful and for making a decision on this form of helpful themes most people are really desperate to be informed on. My sincere regret for not expressing appreciation to sooner.

  39. Hiya, I am really glad I have found this info. Today bloggers publish only about gossips and internet and this is actually irritating. A good web site with exciting content, that is what I need. Thanks for keeping this site, I will be visiting it. Do you do newsletters? Can’t find it.

  40. Thanks , I have just been searching for info about this subject for a long time and yours is the best I have discovered till now. But, what in regards to the conclusion? Are you sure concerning the supply?

  41. I was just searching for this information for some time. After 6 hours of continuous Googleing, at last I got it in your site. I wonder what’s the lack of Google strategy that don’t rank this kind of informative sites in top of the list. Normally the top websites are full of garbage.

  42. I actually wanted to make a brief message to appreciate you for the magnificent ways you are placing at this site. My long internet investigation has at the end been compensated with high-quality facts to exchange with my companions. I ‘d state that that most of us site visitors are undoubtedly fortunate to live in a great place with many outstanding individuals with very helpful principles. I feel rather grateful to have used your weblog and look forward to tons of more cool moments reading here. Thanks again for all the details.

  43. You could certainly see your skills within the paintings you write. The sector hopes for more passionate writers like you who aren’t afraid to say how they believe. All the time go after your heart. “If the grass is greener in the other fellow’s yard – let him worry about cutting it.” by Fred Allen.

  44. You are my intake , I possess few blogs and often run out from to brand.I believe this website has some rattling wonderful info for everyone. “The individual will always be a minority. If a man is in a minority of one, we lock him up.” by Oliver Wendell Holmes.

  45. I needed to create you the very little word to help say thank you once again for your amazing thoughts you’ve featured above. It has been so seriously generous with people like you in giving openly exactly what a number of people would have offered as an e-book in order to make some money for themselves, especially considering that you might well have done it in case you wanted. The advice also worked to be the good way to be sure that other people online have similar passion just like my own to know a whole lot more concerning this problem. I know there are millions of more pleasant opportunities up front for individuals that examine your website.

  46. I am now not positive where you’re getting your information, however great topic. I needs to spend a while finding out more or working out more. Thanks for great information I used to be on the lookout for this info for my mission.

  47. Hey very nice web site!! Guy .. Excellent .. Amazing .. I will bookmark your site and take the feeds additionally…I’m happy to seek out a lot of helpful info here in the submit, we need develop more techniques in this regard, thanks for sharing.

  48. naturally like your web-site but you need to check the spelling on quite a few of your posts. A number of them are rife with spelling problems and I find it very bothersome to tell the reality then again I’ll certainly come back again.

  49. Thank you for some other informative site. The place else could I am getting that kind of info written in such a perfect means? I’ve a challenge that I’m simply now working on, and I’ve been at the glance out for such information.

  50. Someone essentially help to make critically posts I might state. This is the first time I frequented your website page and up to now? I surprised with the research you made to create this actual submit incredible. Excellent job!

  51. Pretty nice post. I just stumbled upon your weblog and wanted to say that I’ve really enjoyed surfing around your blog posts. After all I’ll be subscribing to your feed and I hope you write again very soon!

  52. I would like to voice my passion for your generosity in support of those people that actually need help with this important issue. Your real commitment to passing the message all-around had become especially invaluable and have really allowed associates like me to realize their desired goals. The insightful report signifies this much to me and somewhat more to my fellow workers. Best wishes; from all of us.

  53. I wish to show appreciation to the writer just for rescuing me from this particular situation. Because of browsing throughout the the web and meeting thoughts that were not beneficial, I figured my entire life was well over. Being alive without the answers to the difficulties you have solved through your report is a serious case, and ones that might have in a negative way damaged my entire career if I had not discovered your web page. Your own know-how and kindness in controlling the whole thing was valuable. I’m not sure what I would’ve done if I had not come upon such a thing like this. I’m able to at this time relish my future. Thanks a lot very much for this impressive and result oriented help. I will not think twice to endorse your web site to any person who should have tips about this issue.

  54. I wanted to write a quick word to be able to thank you for these stunning tips and hints you are writing at this site. My time consuming internet research has finally been paid with high-quality points to share with my friends and family. I would assume that we visitors are extremely endowed to live in a useful network with many special individuals with beneficial methods. I feel very much lucky to have seen your web pages and look forward to some more amazing minutes reading here. Thanks once more for a lot of things.

  55. I have not checked in here for a while as I thought it was getting boring, but the last few posts are great quality so I guess I will add you back to my daily bloglist. You deserve it my friend 🙂

  56. Thanks a lot for providing individuals with an exceptionally marvellous possiblity to read critical reviews from this site. It is always very lovely and also stuffed with fun for me personally and my office friends to visit your site at the least thrice in 7 days to learn the fresh tips you will have. And definitely, I am also certainly pleased for the beautiful tricks served by you. Certain 4 areas on this page are really the most impressive I’ve had.

  57. hello!,I like your writing so much! share we keep up a correspondence extra approximately your post on AOL? I need a specialist on this space to solve my problem. Maybe that’s you! Taking a look ahead to peer you.

  58. I am now not positive the place you’re getting your information, but great topic. I must spend some time finding out much more or working out more. Thanks for fantastic information I used to be looking for this information for my mission.

  59. I got what you mean , regards for putting up.Woh I am pleased to find this website through google. “The test and use of a man’s education is that he finds pleasure in the exercise of his mind.” by Carl Barzun.

  60. Hello there, just became alert to your blog through Google, and found that it is really informative. I am going to watch out for brussels. I will be grateful if you continue this in future. Numerous people will be benefited from your writing. Cheers!

发表评论

电子邮件地址不会被公开。 必填项已用*标注