Re: [KFUG] [질문] FreeBSD 5.3 네트워크 성능 문제입니다.

[Pyun YongHyeon <yongari@xxxxxxxxxxx>]

On Fri, Feb 18, 2005 at 05:06:24PM +0900, laim95 wrote:
 > 
 > #mstat -i
 > interrupt                          total       rate
 > irq6: fdc0                             9          0
 > irq8: rtc                        2970558        127
 > irq13: npx0                            1          0
 > irq15: ata1                           48          0
 > irq16: uhci0                     2355294        101
   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
USB 장치를 사용하지 않는다면 kernel설정에서 USB와 관련된 장치를
모두 제거하세요.(uhci, ohci, usb, ehci등) 100interrupts/sec
로 봐서 ghost interrupt로 생각됩니다.

 > irq24: twa0                        23192          0
 > irq28: twa1                        14998          0
 > irq49: em1                       2317339         99
 > irq50: ahd0                         4955          0
 > irq51: ahd1                           15          0
 > irq0: clk                        2320672         99
 > Total                           10007081        431
 > 
 > # dmesg
 > Copyright (c) 1992-2004 The FreeBSD Project.
 > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
 >         The Regents of the University of California. All rights reserved.
 > FreeBSD 5.3-RELEASE #6: Thu Feb 17 16:44:28 KST 2005
 >     onnet@xxxxxxxxxxxxxxxxxxx:/usr/obj/usr/src/sys/ONNET
 > ACPI APIC Table: <A M I  OEMAPIC >
 > Timecounter "i8254" frequency 1193182 Hz quality 0
 > CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2800.12-MHz 686-class CPU)
 >   Origin = "GenuineIntel"  Id = 0xf29  Stepping = 9
 >   Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
 >   Hyperthreading: 2 logical CPUs
 > real memory  = 2147418112 (2047 MB)
 > avail memory = 2100158464 (2002 MB)
 > FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
 >  cpu0 (BSP): APIC ID:  0
 >  cpu1 (AP): APIC ID:  1
 >  cpu2 (AP): APIC ID:  6
 >  cpu3 (AP): APIC ID:  7
   ^^^^^^^^^^^^^^^^^^^^^^^
BIOS에서 HT는 설정을 OFF하세요.

 > ioapic0 <Version 2.0> irqs 0-23 on motherboard
 > ioapic1 <Version 2.0> irqs 24-47 on motherboard
 > ioapic2 <Version 2.0> irqs 48-71 on motherboard
 > npx0: [FAST]
 > npx0: <math processor> on motherboard
 > npx0: INT 16 interface
 > acpi0: <A M I OEMRSDT> on motherboard
 > acpi0: Power Button (fixed)
 > Timecounter "ACPI-safe" frequency 3579545 Hz quality 1000
 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0
 > cpu0: <ACPI CPU> on acpi0
 > cpu1: <ACPI CPU> on acpi0
 > cpu2: <ACPI CPU> on acpi0
 > cpu3: <ACPI CPU> on acpi0
 > pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
 > pci0: <ACPI PCI bus> on pcib0
 > pcib1: <ACPI PCI-PCI bridge> at device 2.0 on pci0
 > pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.P0P2 - AE_NOT_FOUND
 > pci2: <ACPI PCI bus> on pcib1
 > pci2: <base peripheral, interrupt controller> at device 28.0 (no driver attached)
 > pcib2: <ACPI PCI-PCI bridge> at device 29.0 on pci2
 > pci4: <ACPI PCI bus> on pcib2
 > em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0x9000-0x903f mem 0xfe9a0000-0xfe9bffff irq 48 at device 1.0 on pci4
 > em0: Ethernet address: 00:e0:81:2b:56:46
 > em0:  Speed:N/A  Duplex:N/A
 > em1: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port 0x9400-0x943f mem 0xfe9c0000-0xfe9dffff irq 49 at device 1.1 on pci4
 > em1: Ethernet address: 00:e0:81:2b:56:47
 > em1:  Speed:N/A  Duplex:N/A
 > ahd0: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0x9800-0x98ff,0xa000-0xa0ff mem 0xfe9fc000-0xfe9fdfff irq 50 at device 2.0 on pci4
 > ahd0: [GIANT-LOCKED]
 > aic7902: Ultra320 Wide Channel A, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
 > ahd1: <Adaptec AIC7902 Ultra320 SCSI adapter> port 0xa400-0xa4ff,0xa800-0xa8ff mem 0xfe9fe000-0xfe9fffff irq 51 at device 2.1 on pci4
 > ahd1: [GIANT-LOCKED]
 > aic7902: Ultra320 Wide Channel B, SCSI Id=7, PCI-X 67-100Mhz, 512 SCBs
 > pci2: <base peripheral, interrupt controller> at device 30.0 (no driver attached)
 > pcib3: <ACPI PCI-PCI bridge> at device 31.0 on pci2
 > pci3: <ACPI PCI bus> on pcib3
 > 3ware device driver for 9000 series storage controllers, version: 2.50.00.000
 > twa0: <3ware 9000 series Storage Controller> port 0x7400-0x74ff mem 0xfb000000-0xfb7fffff,0xfe6ff800-0xfe6ff8ff irq 24 at device 3.0 on pci3
 > twa0: [GIANT-LOCKED]
 > twa0: 8 ports, Firmware FE9X 2.04.00.005, BIOS BE9X 2.03.01.047
 > twa1: <3ware 9000 series Storage Controller> port 0x7800-0x78ff mem 0xfb800000-0xfbffffff,0xfe6ffc00-0xfe6ffcff irq 28 at device 6.0 on pci3
 > twa1: [GIANT-LOCKED]
 > twa1: 8 ports, Firmware FE9X 2.04.00.005, BIOS BE9X 2.03.01.047
 > pci0: <serial bus, USB> at device 29.0 (no driver attached)
 > pci0: <serial bus, USB> at device 29.1 (no driver attached)
 > pci0: <serial bus, USB> at device 29.7 (no driver attached)
 > pcib4: <ACPI PCI-PCI bridge> at device 30.0 on pci0
 > pci1: <ACPI PCI bus> on pcib4
 > fxp0: <Intel 82551 Pro/100 Ethernet> port 0x6800-0x683f mem 0xfe5a0000-0xfe5bffff,0xfe5fe000-0xfe5fefff irq 17 at device 1.0 on pci1
 > miibus0: <MII bus> on fxp0
 > inphy0: <i82555 10/100 media interface> on miibus0
 > inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
 > fxp0: Ethernet address: 00:e0:81:2b:56:05
 > pci1: <display, VGA> at device 2.0 (no driver attached)
 > isab0: <PCI-ISA bridge> at device 31.0 on pci0
 > isa0: <ISA bus> on isab0
 > atapci0: <Intel ICH5 UDMA100 controller> port 0xfc00-0xfc0f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 31.1 on pci0
 > ata0: channel #0 on atapci0
 > ata1: channel #1 on atapci0
 > pci0: <serial bus, SMBus> at device 31.3 (no driver attached)
 > acpi_button0: <Power Button> on acpi0
 > sio0: configured irq 4 not in bitmap of probed irqs 0
 > sio0: port may not be enabled
 > sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
 > sio0: type 16550A
 > fdc0: <floppy drive controller (FDE)> port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0
 > fdc0: [FAST]
 > fd0: <1440-KB 3.5" drive> on fdc0 drive 0
 > sio1: configured irq 3 not in bitmap of probed irqs 0
 > sio1: port may not be enabled
 > sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0
 > sio1: type 16550A
 > orm0: <ISA Option ROMs> at iomem 0xc9800-0xc9fff,0xc8000-0xc97ff,0xc0000-0xc7fff on isa0
 > pmtimer0 on isa0
 > atkbdc0: <Keyboard controller (i8042)> at port 0x64,0x60 on isa0
 > atkbd0: <AT Keyboard> irq 1 on atkbdc0
 > kbd0 at atkbd0
 > atkbd0: [GIANT-LOCKED]
 > ppc0: parallel port not found.
 > sc0: <System console> at flags 0x100 on isa0
 > sc0: VGA <16 virtual consoles, flags=0x300>
 > vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
 > Timecounters tick every 10.000 msec
 > acpi_cpu: throttling enabled, 8 steps (100% to 12.5%), currently 100.0%
 > acd0: CDROM <SAMSUNG CD-ROM SH-152A/C505> at ata1-master UDMA33
 > Waiting 15 seconds for SCSI devices to settle
 > da0 at ahd0 bus 0 target 0 lun 0
 > da0: <SEAGATE ST336607LW 0006> Fixed Direct Access SCSI-3 device 
 > da0: 320.000MB/s transfers (160.000MHz, offset 63, 16bit), Tagged Queueing Enabled
 > da0: 35003MB (71687372 512 byte sectors: 255H 63S/T 4462C)
 > da1 at twa0 bus 0 target 0 lun 0
 > da1: <3ware Logical Disk 00 1.00> Fixed Direct Access SCSI-0 device 
 > da1: 100.000MB/s transfers
 > da1: 1430448MB (2929557504 512 byte sectors: 255H 63S/T 182356C)
 > da2 at twa1 bus 0 target 0 lun 0
 > da2: <3ware Logical Disk 00 1.00> Fixed Direct Access SCSI-0 device 
 > da2: 100.000MB/s transfers
 > da2: 1430448MB (2929557504 512 byte sectors: 255H 63S/T 182356C)
 > SMP: AP CPU #2 Launched!
 > SMP: AP CPU #1 Launched!
 > SMP: AP CPU #3 Launched!
 > Mounting root from ufs:/dev/da0s1a
 > uhci0: <Intel 82801EB (ICH5) USB controller USB-A> port 0xc400-0xc41f irq 16 at device 29.0 on pci0
 > uhci0: [GIANT-LOCKED]
 > usb0: <Intel 82801EB (ICH5) USB controller USB-A> on uhci0
 > usb0: USB revision 1.0
 > uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 > uhub0: 2 ports with 2 removable, self powered
 > uhci1: <Intel 82801EB (ICH5) USB controller USB-B> port 0xc800-0xc81f irq 19 at device 29.1 on pci0
 > uhci1: [GIANT-LOCKED]
 > usb1: <Intel 82801EB (ICH5) USB controller USB-B> on uhci1
 > usb1: USB revision 1.0
 > uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 > uhub1: 2 ports with 2 removable, self powered
 > em1: Link is up 100 Mbps Full Duplex
 > ehci0: <EHCI (generic) USB 2.0 controller> mem 0xfebffc00-0xfebfffff irq 23 at device 29.7 on pci0
 > ehci0: [GIANT-LOCKED]
 > ehci_pci_attach: companion usb0
 > ehci_pci_attach: companion usb1
 > usb2: waiting for BIOS to give up control
 > usb2: timed out waiting for BIOS
 > usb2: EHCI version 1.0
 > usb2: companion controllers, 2 ports each: usb0 usb1
 > usb2: <EHCI (generic) USB 2.0 controller> on ehci0
 > usb2: USB revision 2.0
 > uhub2: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
 > uhub2: 4 ports with 4 removable, self powered
 > 
 > > [...]
 > > > > 전반적으로 5.3R은 4.x보다는 조금 느리지만 SMP에서는 대부분의 경우 
 > > > > 더 빠릅니다. Local network에서 시험하는게 아니라면 TCP SACK을
 > > > > disable해보세요.
 > > > > 
 > > 
 > > 이 질문에 답하지 않으셨습니다. 시스템이 clinet요청을 처리할때 ps등으로
 > > 시스템의 상태를 확인해 보셨나요?
 > > 시험한 client는 무엇으로 검사했나요? 서버시스템이 idle상태에서도 속도의
 > > 변화가 있다는 말인가요?
 > 
 > 클라이언트 프로그램은 저희가 windows 용으로 자체 개발한 툴을 사용중입니다.
 > ps로 확인결과 아파치만 정상적으로 떠있습니다.전에 알려 들였듯이 아파치 기반 cgi 프로그램이
 > get/put 작업을 진행합니다.

ps등으로 보면 해당 CGI가 어떤 STATE에 있나요?

 > idle 시에 모든접속을 막고 test용 1대만 접속하여 put/get 작업을 진행하였을때도
 > 현저하게 속도 차이가 납니다.
속도차이에 대해서 좀 더 자세히 말씀해주세요.
SACK을 disable해보셨나요?(#sysctl net.inet.tcp.sack.enable=0)
중간에 방화벽이 있나요?
WinXP의 경우는 SACK구현에 잘못된 부분이 있어서 만일 제대로 된 stateful
inspection을 하는 방화벽이 중간에 있다면 WinXP의 packet을 drop할 수
있습니다.(이 경우 속도 저하로 나타납니다. 4.x는 TCP SACK을 지원하지
않습니다.)

CGI가 아닌 FTP같은것으로도 속도 차이를 느끼시나요?

 > sata raid의 경우 하드수가 들어난다고 file I/O에 차이를 보이는건지 모르겠습니다.
 > 
차이는 있을 수 있지만 현저한 차이는 느낄 수 없는게 정상이라 생각합니다.

PS. RAID0를 제외한 모든 모드는 RAID를 사용하지 않을때보다도 느립니다.
-- 
Regards,
Pyun YongHyeon
http://www.kr.freebsd.org/~yongari	|	yongari@xxxxxxxxxxx
_______________________________________________
한국 FreeBSD 사용자 그룹(KFUG) questions 메일링 리스트
questions at kr.FreeBSD.org
http://www.kr.FreeBSD.org/mailman/listinfo/questions