RE: Cisco Nexus vs Juniper QFabric from Joseph L. Brunner on 2012-07-13 (Ccielab archives 07/2012)

From: Joseph L. Brunner <joe_at_affirmedsystems.com>
Date: Fri, 13 Jul 2012 09:30:09 +0000

Best post in a long time b thank you so so muchb&

If you were a defense lawyer with those arguments b I would be walking out of court with an acquittal!

Good work. Definitely printing this and keeping it laminated

From: Guillermo Ruiz [mailto:gruizesteban_at_gmail.com]
Sent: Friday, July 13, 2012 5:23 AM
To: Taufik Kurniawan
Cc: Alexei Monastyrnyi; Joseph L. Brunner; muhammad.nasim_at_gmail.com; sonubi02_at_gmail.com; ccielab_at_groupstudy.com
Subject: Re: Cisco Nexus vs Juniper QFabric

I do not have any experience in Arista...but when compared with Cisco and Juniper it needs to be considered a serious candidate. Here is a small comparision between Juniper and Cisco based in 3 basic elements, scalability, power consumption and physical deployment. Sorry for the long post...

o;?Juniper is intended to be a One Network, flat and any to any connectivity. It has 5us or less latency (new Cisco F2 modules are running 6us - and are still being validated), a single point of management and runs a single version of JunOS (far away from all those different versions of IOS n Cisco). Looks really nice...

The architecture is divided into 3 blocks:

- QFabric Director: QFX3100 (Requires minimum 2 x Directors)

- QFabric Nodes: QFX3500 (Max. 128 nodes)

- QFabric Interconnect: QFX 3008 (Max. 4 QF Interconnects)

Let's consider three basic parameters: Physical deployment, scalability and power consumption.

1) One flat Network: No, not really

If you physically deploy the Q-Fabric you will find that it is pretty similar to build a Nexus 7K to N5K architecture (spine-leaf topology). You will require the same topology...

A funny tip is that QFabric uses Broadcom BCM56840 Series, known as Trident....and it's the same ASIC used in NEXUS 3000!!! At least these is used for HPC ;-)

2) Scalability

Some numbers on this topic. Bear in mind that the Broadcom ASIC consumes one switch ID:

B7 QFabric Nodes: Maximum of 128 x QFX3500, that makes a total of 128 switch IDs (there is one ASIC per QFX3500)
B7 QFabric Interconnect consumes 24 switch IDs per chassis. Supporting a maximum of 4x QF/Interconnects, it gives: 96 switch IDs

- Each Front card consumes 2 switch IDs (2xASICs per front linecard): 8 slots x 2= 16

- Each Rear card consumes 1 switch ID (1xASIC per rear linecard): 8 x 1= 8

The maximum number of switch IDs is determined by the Broadcom header of the ethernet frame, which is an 8 bit address space: 2^8= 256 switch IDs, thus we will have 256 - 128- 96 = 32 IDs left

B?B?B?32 ??? Oh dear, this does not look scalable enough for some large data centres.

3) Power / Heat dissipation

Let's take the official datasheets as Juniper does not have an online power calculator as Cisco (which provides a real power consumption). Note that real power consumption and heat dissipation will depend on number of linecards and ports being used. So this is a theoretical calculation based in vendor's datasheet.

Device

Power

Heat Dissipation

QFX3008 (Fabric Interconnect)

5.2KW

17.750 BTU/hr

QFX3100 (Fabric Director)

476W

1.624 BTU/hr

QFX3500 (Fabric Node)

365W

1.250 BTU/hr

Nexus 7010

4.7KW

17.200 BTU/hr

Nexus 5548UP

390W

1.998 BTU/hr

Minimum Juniper Configuration: 2 x Fabric Interconnect, 2 x QFabric Directors, 4 x QFabric Nodes

Power: 12.82 KW

Heat: 43.748 BTU/hr

Minimum Cisco Nexus Configuration: 2 x Nexus 7010, 4 x Nexus 5548UP

Power: 10.96 KW

Heat dissipation: 42.392 BTU/hr

As shown above, Cisco has lower power and heat dissipation than a QFabric deployment.

And what about the cost of the QFabric? Is it as expensive as a Nexus solution?

Here is a kit list. As you will see there is no discount applied as this is a brand new technology. Maybe I get wrong, but do not expect a 48-60% discount as you can find with Cisco.

o;?

Item

Part Code

Description

Qty

Unit List

Total List

% Discount

Reseller Buy

Kitlist

QFX3008-FAB-2INTC-BNDL

QFabric QF/Interconnect Bundle with 2 QF/Interconnects, 2 16-port 40G I/O modules and 16 QFX-SFP-1GE-T optics (Note: AC power cords are sold separately)

$470.000

$ 470.000,00

QFX3500-48S4Q-ACR

QFX3500, 48 SFP+/SFP and 4 QSFP ports, redundant dual AC power supply, front to back air flow.

$34.000

$ 204.000,00

QFX-QSFP-40G-SR4

QSFP+ 40GBase-SR4 40 Gigabit Optics, 850nm for up to 150m transmission on MMF

$5.000

$ 240.000,00

QFX3008-FAB-CPE-BNDL

QFabric control plane ethernet bundle with 8 EX4200-48T, 8 uplink modules and 16 EX-SFP-10GE-USR optics (Note: AC power cords are sold separately)

$102.400

$ 102.400,00

QFX3008-FAB-DRCTR-BNDL

QFabric QF/Director bundle with 2 QF/Directors (Note: AC power cords are sold separately)

$45.000

$ 45.000,00

QFXC08-CABMAN

Cable management module for QFXC08 chassis

$500

$ 1.000,00

QFXC08-ACTRAY-D

A/C Power Wiring Tray (Three phase Delta) for QFXC08 Chassis

$2.000

$ 8.000,00

CBL-EX-PWR-C13-EU

AC Power Cable, Europe (10A/250V, 2.5meter)

$50

$ 400,00

QFX3008-JSL-DRCTR-FAB

QFX3008 Series Base Fabric Software

$50.000

$ 50.000,00

QFX3000-JSL-EDGE-FAB

QFX3000 Series QF/Node featurelicense

$5.000

$ 30.000,00

QFX-JSL-DRCTR-ADV1

QFabric Advanced feature license for IS-IS, BGP and IPv6 Routing (per 8 nodes)

$10.000

$ 0,00

QFX-JSL-EDGE-ADV1

QFX3500 Series edge Advanced feature license for IS-IS, BGP, IPv6 Routing

$5.000

$ 0,00

(**) 3 options for A/C Power Wiring tray: 3 phase Delta, 3 phase WYE or single phase -- change according to customer requirements

Subtotal for 1 Qfabric

$ 1.150.800,00

CONCLUSIONS

The drawbacks I can see:

1. QFabric is not compatible with other Juniper devices (e.g. M/MX/EX/SRX/T/J Series and more...). It only works with QFX3500.
2. Requires more cabling than a normal Nexus deployment (you have to add connections to the QF/Director while Nexus has the SUP modules within the chassis). Even if we need to add dedicated point to point L3 connection to build adjacencies in the Nexus environment, the number is still lower.
3. Requires more devices than Nexus
4. Consumes more power than a Nexus infrastructure
5. There is currently few LIVE networks in customer (as opposed to Nexus deployments). Majority of deployments are in testing and development environments.

Benefits

The big benefit I can see for this is still in the roadmap. It's expected to have a new software release in September 2012 allowing 4 x QF/Interconnect to be interconnected as a single logical unit. Currently only 2 x QF/Interconnect can be connected. This will become a very nice solution to allow L2 between DC (if the distance is not higher than 40 Km). Merge 2 DC into a single logical domain can bring a lot of benefits for VMotion, mainframe applications, etc. (and some headaches for network engineers - but hey! who said life was easy?). Cisco's alternative is OTV (which requires a dedicated VDC and in some cases you might run out of VDCs). There are some workarounds to this like getting the new SUP-2E modules which scales up to 8 VDCs, but it requires additional license cost to allow this 4 additional VDCs...or you can go with a pair of new Nexus 7004 (but again it will increase your cost).

You can always ask Juniper to take a walk through their LAB in Amsterdam and have a look to the Q-Fabric ;-)

Guillermo

Blogs and organic groups at http://www.ccie.net
Received on Fri Jul 13 2012 - 09:30:09 ART

This archive was generated by hypermail 2.2.0 : Wed Aug 01 2012 - 15:55:23 ART