Measure startup time
The NetBSD Project
Status: Closed
Time to complete: 96 hrs
Mentors: Radoslaw Kujawa, Aleksej Saushev, Julian Coleman, Julian Fagir
NetBSD doesn't have many services enabled by default, though most people add some more services they need for a system (depending on the usage).
Your task is about researching some of the most used services and the daemons that are inside NetBSD already, and measure the startup time depending on which and how many services are enabled.
You should write your results either in tabular form or in a paper at least two pages long, then also concluding your results.
Uploaded Work
| File name/URL | File size | Date submitted | |
|---|---|---|---|
| boottime.pdf | 48.5 KB | December 17 2012 19:40 UTC | |
| boottime.tex | 2.0 KB | December 17 2012 21:24 UTC | |
| boottime.pdf | 70.2 KB | December 18 2012 19:52 UTC | |
| boottime.tex | 4.9 KB | December 18 2012 19:52 UTC | |
| boottime.pdf | 70.2 KB | December 18 2012 20:10 UTC | |
| boottime.tex | 4.9 KB | December 18 2012 20:10 UTC |
I would like to work on this task.
This task has been assigned to mgoszcz2. You have 96 hours to complete this task, good luck!
Hi!
This task is not easy one and subject to some controversy.
Please, try to reach us on #netbsd-code to discuss progress.
If you're misdirected or stuck, we need to get you on track as soon as possible.
I'm sorry but I over the last few days I didn't had internet acess for long enough to work on the task. Also my attempts to get netBsd working are failing so it's proably better if you reopen the task :(
The claim on this task has been removed, someone else can claim it now.
I would like to work on this task.
The claim on this task has been removed, someone else can claim it now.
I would like to work on this task.
This task has been assigned to Michał Zieliński. You have 96 hours to complete this task, good luck!
Hi!
You're working on the task for 3 days, by this time you should have elaborated at least your approach to the problem and get some results. Could you submit your preliminary for review?
I have measured startup times on virtual machine. They were very similar, regardless of number of enabled services.
Today I'm measuring startup on real machine.
I am measuring starup time by simple script which ssh's into target machine, reboots it and waits for reboot to complete. I regard startup as complete when init invokes /etc/rc.local.
I would upload results, but I left them in my home on computer which is turned off.
The work on this task is ready to be reviewed.
Hi!
The report as you provide it cannot be taken seriously. It lacks description of method (including illustrations how to set experiment up and how to run it), thus it is hard to reproduce. It lacks description of how much time is spent on each stage. Also, measuring time on virtual hardware makes little sense, unless you gather a lot of samples since dispersion is very high in these cases. If VM is the only "hardware" available to you, how many times have you run your measurements? What is the distribution?
Your report lacks the most important analysis of data acquired. What fraction of time is spent in BIOS? loader? kernel? rc? How much time is spent in sh? Can we draw any conclusions from the report? Does startup mechanism need to be improved or is it impractical?
One of the mentors has sent this task back for more work. Talk to the mentor(s) assigned to this task to satisfy the requirements needed to complete this task, submit your work again and mark the task as complete once you re-submit your work.
The deadline of the task has been extended with 2 days and 0 hours.
I wonder if you are commenting the same PDF file as I have uploaded. I have explicitly written that I used virtual machine and real PC. I have also mentioned that I repeated tests 10 times. I have also written about my approach, but I felt that there is no reason to include 20 line bash script and inform how to connect ethernet cable to computer.
As far as I know NetBSD doesn't have any distributions. At least netbsd.org page doesn't offer anything more than one image for each architecture, format and version.
The second paragraph makes more sense to me - I will get more detailed data and analysis. I will also expand sentence about BIOS, DHCP and kernel taking majority of time, to conclude that rc is good enough.
Michal, it's good that you've understood the second paragraph, however about the first paragraph:
Aleksej expressed valid concerns regarding your PDF. The problem is that you didn't use statistical methods (or at least didn't describe it). Do you know anything about statistics? Maybe it's time to learn, as this knowledge is very useful in a task where you have to measure performance of something. The words "samples", "dispersion" and "distribution" were used in statistical sense. Until you understand what they mean then first paragraph won't make much sense to you.
Just measuring average boot time is not enough.
Oh, I needed to switch from IT mode to physics mode.
I hope that standard deviation from average will be enough.
Hi!
There's fundamental misunderstanding on your side. The inclusion of 20-line bash script and description of test setup is the most important part here. It is the basis of your approach to measurement, it cannot be unimportant
For instance, you could have chosen the most straightforward approach with sitting with stop-watch and waiting for messages to appear on console. Though it sounds stupid (and it is, there're better approaches), it is _perfectly_valid_. Only it requires separate experiments, including ones to measure your latency.
I strongly suggest that you don't throw away unprocessed measurements. Standard deviation makes sense for normal distribution, and it isn't clear yet whether yours is. You don't need excellent knowledge of statistics to draw a picture with dots surrounding some mean value, but you do need to understand some basics.
In general, your report should give to another person clear understanding what one should do to repeat your experiments, get similar measurements, and make conclusions similar to what you make.
The work on this task is ready to be reviewed.
Thanks, good work! I'm closing this task.
Congratulations, this task has been completed successfully.
I have just noticed that I did "*0.5" instead of "**0.5" when calculating std deviation.
http://user.zielm.com/gci/